Virtual filesystem component¶
Overview¶
Virtual filesystem (VFS) component provides a unified interface for drivers which can perform operations on file-like objects. This can be a real filesystems (FAT, SPIFFS, etc.), or device drivers which exposes file-like interface.
This component allows C library functions, such as fopen and fprintf, to work with FS drivers. At high level, each FS driver is associated with some path prefix. When one of C library functions needs to open a file, VFS component searches for the FS driver associated with the file’s path, and forwards the call to that driver. VFS also forwards read, write, and other calls for the given file to the same FS driver.
For example, one can register a FAT filesystem driver with /fat
prefix, and call fopen("/fat/file.txt", "w")
. VFS component will then call open
function of FAT driver and pass /file.txt
argument to it (and appropriate mode flags). All subsequent calls to C library functions for the returned FILE*
stream will also be forwarded to the FAT driver.
FS registration¶
To register an FS driver, application needs to define in instance of esp_vfs_t structure and populate it with function pointers to FS APIs:
esp_vfs_t myfs = {
.flags = ESP_VFS_FLAG_DEFAULT,
.write = &myfs_write,
.open = &myfs_open,
.fstat = &myfs_fstat,
.close = &myfs_close,
.read = &myfs_read,
};
ESP_ERROR_CHECK(esp_vfs_register("/data", &myfs, NULL));
Depending on the way FS driver declares its APIs, either read
, write
, etc., or read_p
, write_p
, etc. should be used.
Case 1: API functions are declared without an extra context pointer (FS driver is a singleton):
ssize_t myfs_write(int fd, const void * data, size_t size);
// In definition of esp_vfs_t:
.flags = ESP_VFS_FLAG_DEFAULT,
.write = &myfs_write,
// ... other members initialized
// When registering FS, context pointer (third argument) is NULL:
ESP_ERROR_CHECK(esp_vfs_register("/data", &myfs, NULL));
Case 2: API functions are declared with an extra context pointer (FS driver supports multiple instances):
ssize_t myfs_write(myfs_t* fs, int fd, const void * data, size_t size);
// In definition of esp_vfs_t:
.flags = ESP_VFS_FLAG_CONTEXT_PTR,
.write_p = &myfs_write,
// ... other members initialized
// When registering FS, pass the FS context pointer into the third argument
// (hypothetical myfs_mount function is used for illustrative purposes)
myfs_t* myfs_inst1 = myfs_mount(partition1->offset, partition1->size);
ESP_ERROR_CHECK(esp_vfs_register("/data1", &myfs, myfs_inst1));
// Can register another instance:
myfs_t* myfs_inst2 = myfs_mount(partition2->offset, partition2->size);
ESP_ERROR_CHECK(esp_vfs_register("/data2", &myfs, myfs_inst2));
Paths¶
Each registered FS has a path prefix associated with it. This prefix may be considered a “mount point” of this partition.
In case when mount points are nested, the mount point with the longest matching path prefix is used when opening the file. For instance, suppose that the following filesystems are registered in VFS:
- FS 1 on /data
- FS 2 on /data/static
Then:
- FS 1 will be used when opening a file called
/data/log.txt
- FS 2 will be used when opening a file called
/data/static/index.html
- Even if
/index.html"
doesn’t exist in FS 2, FS 1 will not be searched for/static/index.html
.
As a general rule, mount point names must start with the path separator (/
) and must contain at least one character after path separator. However an empty mount point name is also supported, and may be used in cases when application needs to provide “fallback” filesystem, or override VFS functionality altogether. Such filesystem will be used if no prefix matches the path given.
VFS does not handle dots (.
) in path names in any special way. VFS does not treat ..
as a reference to the parent directory. I.e. in the above example, using a path /data/static/../log.txt
will not result in a call to FS 1 to open /log.txt
. Specific FS drivers (such as FATFS) may handle dots in file names differently.
When opening files, FS driver will only be given relative path to files. For example:
myfs
driver is registered with/data
as path prefix- and application calls
fopen("/data/config.json", ...)
- then VFS component will call
myfs_open("/config.json", ...)
. myfs
driver will open/config.json
file
VFS doesn’t impose a limit on total file path length, but it does limit FS path prefix to ESP_VFS_PATH_MAX
characters. Individual FS drivers may have their own filename length limitations.
File descriptors¶
It is suggested that filesystem drivers should use small positive integers as file descriptors. VFS component assumes that CONFIG_MAX_FD_BITS
bits (12 by default) are sufficient to represent a file descriptor.
While file descriptors returned by VFS component to newlib library are rarely seen by the application, the following details may be useful for debugging purposes. File descriptors returned by VFS component are composed of two parts: FS driver ID, and the actual file descriptor. Because newlib stores file descriptors as 16-bit integers, VFS component is also limited by 16 bits to store both parts.
Lower CONFIG_MAX_FD_BITS
bits are used to store zero-based file descriptor. The per-filesystem FD obtained from the FS open
call, and this result is stored in the lower bits of the FD. Higher bits are used to save the index of FS in the internal table of registered filesystems.
When VFS component receives a call from newlib which has a file descriptor, this file descriptor is translated back to the FS-specific file descriptor. First, higher bits of FD are used to identify the FS. Then only the lower CONFIG_MAX_FD_BITS
bits of the fd are masked in, and resulting FD is passed to the FS driver.
FD as seen by newlib FD as seen by FS driver
+-------+---------------+ +------------------------+
| FS id | Zero—based FD | +--------------------------> |
+---+---+------+--------+ | +------------------------+
| | |
| +--------------+
|
| +-------------+
| | Table of |
| | registered |
| | filesystems |
| +-------------+ +-------------+
+-------> entry +----> esp_vfs_t |
index +-------------+ | structure |
| | | |
| | | |
+-------------+ +-------------+
Standard IO streams (stdin, stdout, stderr)¶
If “UART for console output” menuconfig option is not set to “None”, then stdin
, stdout
, and stderr
are configured to read from, and write to, a UART. It is possible to use UART0 or UART1 for standard IO. By default, UART0 is used, with 115200 baud rate, TX pin is GPIO1 and RX pin is GPIO3. These parameters can be changed in menuconfig.
Writing to stdout
or stderr
will send characters to the UART transmit FIFO. Reading from stdin
will retrieve characters from the UART receive FIFO.
By default, VFS uses simple functions for reading from and writing to UART. Writes busy-wait until all data is put into UART FIFO, and reads are non-blocking, returning only the data present in the FIFO. Because of this non-blocking read behavior, higher level C library calls, such as fscanf("%d\n", &var);
may not have desired results.
Applications which use UART driver may instruct VFS to use the driver’s interrupt driven, blocking read and write functions instead. This can be done using a call to esp_vfs_dev_uart_use_driver
function. It is also possible to revert to the basic non-blocking functions using a call to esp_vfs_dev_uart_use_nonblocking
.
VFS also provides optional newline conversion feature for input and output. Internally, most applications send and receive lines terminated by LF (‘’n’‘) character. Different terminal programs may require different line termination, such as CR or CRLF. Applications can configure this separately for input and output either via menuconfig, or by calls to esp_vfs_dev_uart_set_rx_line_endings
and esp_vfs_dev_uart_set_tx_line_endings
functions.
Standard streams and FreeRTOS tasks¶
FILE
objects for stdin
, stdout
, and stderr
are shared between all FreeRTOS tasks, but the pointers to these objects are are stored in per-task struct _reent
. The following code:
fprintf(stderr, "42\n");
actually is translated to to this (by the preprocessor):
fprintf(__getreent()->_stderr, "42\n");
where the __getreent()
function returns a per-task pointer to struct _reent
(newlib/include/sys/reent.h#L370-L417). This structure is allocated on the TCB of each task. When a task is initialized, _stdin
, _stdout
and _stderr
members of struct _reent
are set to the values of _stdin
, _stdout
and _stderr
of _GLOBAL_REENT
(i.e. the structure which is used before FreeRTOS is started).
Such a design has the following consequences:
- It is possible to set
stdin
,stdout
, andstderr
for any given task without affecting other tasks, e.g. by doingstdin = fopen("/dev/uart/1", "r")
. - Closing default
stdin
,stdout
, orstderr
usingfclose
will close theFILE
stream object — this will affect all other tasks. - To change the default
stdin
,stdout
,stderr
streams for new tasks, modify_GLOBAL_REENT->_stdin
(_stdout
,_stderr
) before creating the task.
Application Example¶
API Reference¶
Header File¶
Functions¶
-
ssize_t
esp_vfs_write
(struct _reent *r, int fd, const void *data, size_t size)¶ These functions are to be used in newlib syscall table. They will be called by newlib when it needs to use any of the syscalls.
-
off_t
esp_vfs_lseek
(struct _reent *r, int fd, off_t size, int mode)¶
-
ssize_t
esp_vfs_read
(struct _reent *r, int fd, void *dst, size_t size)¶
-
int
esp_vfs_open
(struct _reent *r, const char *path, int flags, int mode)¶
-
int
esp_vfs_close
(struct _reent *r, int fd)¶
-
int
esp_vfs_fstat
(struct _reent *r, int fd, struct stat *st)¶
-
int
esp_vfs_stat
(struct _reent *r, const char *path, struct stat *st)¶
-
int
esp_vfs_link
(struct _reent *r, const char *n1, const char *n2)¶
-
int
esp_vfs_unlink
(struct _reent *r, const char *path)¶
-
int
esp_vfs_rename
(struct _reent *r, const char *src, const char *dst)¶
-
esp_err_t
esp_vfs_register
(const char *base_path, const esp_vfs_t *vfs, void *ctx)¶ Register a virtual filesystem for given path prefix.
- Return
- ESP_OK if successful, ESP_ERR_NO_MEM if too many VFSes are registered.
- Parameters
base_path
: file path prefix associated with the filesystem. Must be a zero-terminated C string, up to ESP_VFS_PATH_MAX characters long, and at least 2 characters long. Name must start with a “/” and must not end with “/”. For example, “/data” or “/dev/spi” are valid. These VFSes would then be called to handle file paths such as “/data/myfile.txt” or “/dev/spi/0”.vfs
: Pointer to esp_vfs_t, a structure which maps syscalls to the filesystem driver functions. VFS component doesn’t assume ownership of this pointer.ctx
: If vfs->flags has ESP_VFS_FLAG_CONTEXT_PTR set, a pointer which should be passed to VFS functions. Otherwise, NULL.
-
esp_err_t
esp_vfs_register_socket_space
(const esp_vfs_t *vfs, void *ctx, int *p_min_fd, int *p_max_fd)¶ Special case function for registering a VFS that uses a method other than open() to open new file descriptors.
This is a special-purpose function intended for registering LWIP sockets to VFS.
- Return
- ESP_OK if successful, ESP_ERR_NO_MEM if too many VFSes are registered.
- Parameters
vfs
: Pointer to esp_vfs_t. Meaning is the same as for esp_vfs_register().ctx
: Pointer to context structure. Meaning is the same as for esp_vfs_register().p_min_fd
: If non-NULL, on success this variable is written with the minimum (global/user-facing) FD that this VFS will use. This is useful when ESP_VFS_FLAG_SHARED_FD_SPACE is set in vfs->flags.p_max_fd
: If non-NULL, on success this variable is written with one higher than the maximum (global/user-facing) FD that this VFS will use. This is useful when ESP_VFS_FLAG_SHARED_FD_SPACE is set in vfs->flags.
-
esp_err_t
esp_vfs_unregister
(const char *base_path)¶ Unregister a virtual filesystem for given path prefix
- Return
- ESP_OK if successful, ESP_ERR_INVALID_STATE if VFS for given prefix hasn’t been registered
- Parameters
base_path
: file prefix previously used in esp_vfs_register call
Structures¶
-
struct
esp_vfs_t
¶ VFS definition structure.
This structure should be filled with pointers to corresponding FS driver functions.
VFS component will translate all FDs so that the filesystem implementation sees them starting at zero. The caller sees a global FD which is prefixed with an pre-filesystem-implementation.
Some FS implementations expect some state (e.g. pointer to some structure) to be passed in as a first argument. For these implementations, populate the members of this structure which have _p suffix, set flags member to ESP_VFS_FLAG_CONTEXT_PTR and provide the context pointer to esp_vfs_register function. If the implementation doesn’t use this extra argument, populate the members without _p suffix and set flags member to ESP_VFS_FLAG_DEFAULT.
If the FS driver doesn’t provide some of the functions, set corresponding members to NULL.
Public Members
-
int
flags
¶ ESP_VFS_FLAG_CONTEXT_PTR or ESP_VFS_FLAG_DEFAULT, plus optionally ESP_VFS_FLAG_SHARED_FD_SPACE
-
int
Macros¶
-
ESP_VFS_PATH_MAX
¶ Maximum length of path prefix (not including zero terminator)
-
ESP_VFS_FLAG_CONTEXT_PTR
¶ Flag which indicates that FS needs extra context pointer in syscalls.
-
ESP_VFS_FLAG_SHARED_FD_SPACE
¶ Flag which indicates that the FD space of the VFS implementation should be made the same as the FD space in newlib. This means that the normal masking off of VFS-independent fd bits is ignored and the full user-facing fd is passed to the VFS implementation.
Set the p_minimum_fd & p_maximum_fd pointers when registering the socket in order to know what range of FDs can be used with the registered VFS.
This is mostly useful for LWIP which shares the socket FD space with socket-specific functions.
Header File¶
Functions¶
-
void
esp_vfs_dev_uart_register
()¶ add /dev/uart virtual filesystem driver
This function is called from startup code to enable serial output
-
void
esp_vfs_dev_uart_set_rx_line_endings
(esp_line_endings_t mode)¶ Set the line endings expected to be received on UART.
This specifies the conversion between line endings received on UART and newlines (‘
’, LF) passed into stdin:
- ESP_LINE_ENDINGS_CRLF: convert CRLF to LF
- ESP_LINE_ENDINGS_CR: convert CR to LF
- ESP_LINE_ENDINGS_LF: no modification
- Note
- this function is not thread safe w.r.t. reading from UART
- Parameters
mode
: line endings expected on UART
-
void
esp_vfs_dev_uart_set_tx_line_endings
(esp_line_endings_t mode)¶ Set the line endings to sent to UART.
This specifies the conversion between newlines (‘
’, LF) on stdout and line endings sent over UART:
- ESP_LINE_ENDINGS_CRLF: convert LF to CRLF
- ESP_LINE_ENDINGS_CR: convert LF to CR
- ESP_LINE_ENDINGS_LF: no modification
- Note
- this function is not thread safe w.r.t. writing to UART
- Parameters
mode
: line endings to send to UART
-
void
esp_vfs_dev_uart_use_nonblocking
(int uart_num)¶ set VFS to use simple functions for reading and writing UART Read is non-blocking, write is busy waiting until TX FIFO has enough space. These functions are used by default.
- Parameters
uart_num
: UART peripheral number
-
void
esp_vfs_dev_uart_use_driver
(int uart_num)¶ set VFS to use UART driver for reading and writing
- Note
- application must configure UART driver before calling these functions With these functions, read and write are blocking and interrupt-driven.
- Parameters
uart_num
: UART peripheral number