Index filesystem structure

Index filesystem structure

A description of the index filesystem structure.

All index files live within a subdirectory of the data directory named @indexes . Within this subdirectory, there’s a subdirectory for each bucket (which matches exactly the bucket name).

Any index file has the form <type>_<hexadecimal_signature>.view.N Each component’s meaning is:

  • type - the index type, can be main (active vbuckets data) or replica (replica vbuckets data)

  • hexadecimal_signature - this is the hexadecimal form of an MD5 hash computed over the map/reduce functions of a design document, when these functions change, a new index is created. It’s possible to have multiple versions of the same design document alive (different signatures). This happens for a short period, for example a client does a stale=false request to an index (1 index == 1 design document), which triggers an index build/update and before this update/build finishes, the design document is updated (with different map/reduce functions). The initial version of the index will remain alive until all currently blocked clients on it are served. In the meanwhile new query requests are redirected to the latest (second) version of the index, always. This is what makes it possible to have multiple versions of the same design document index files at any point in time (however for short periods).

  • N - when an index file is created N is 1, always. Every time the index file is compacted, N is incremented by 1. This is similar to what happens for vbucket database files Data missing in query response or it’s wrong (potentially due to server issues)).

For each design document, there’s also a subdirectory named like tmp_<hexadecimal_signature>_<type> . This is a directory containing temporary files used for the initial index build (and soon for incremental optimizations). Files within this directory have a name formed by the design document signature and a generated UUID. These files are periodically deleted when they’re not useful anymore.

All views defined within a design document are backed by a btree data structure, and they all live inside the same index file. Therefore for each design document, independently of the number of views it defines, there’s 2 files, one for main data and the other for replica data.

Example:

> tree couch/0/\@indexes/ couch/0/@indexes/ ??? default ??? main_018b83ca22e53e14d723ea858ba97168.view.1 ??? main_15e1f576bc85e3e321e28dc883c90077.view.1 ??? main_440b0b3ded9d68abb559d58b9fda3e0a.view.1 ??? main_4995c136d926bdaf94fbe183dbf5d5aa.view.1 ??? main_fd2bdf6191e61af6e801e3137e2f1102.view.1 ??? replica_018b83ca22e53e14d723ea858ba97168.view.1 ??? replica_15e1f576bc85e3e321e28dc883c90077.view.1 ??? replica_440b0b3ded9d68abb559d58b9fda3e0a.view.1 ??? replica_4995c136d926bdaf94fbe183dbf5d5aa.view.1 ??? replica_fd2bdf6191e61af6e801e3137e2f1102.view.1 ??? tmp_018b83ca22e53e14d723ea858ba97168_main ??? tmp_15e1f576bc85e3e321e28dc883c90077_main ??? tmp_440b0b3ded9d68abb559d58b9fda3e0a_main ??? tmp_4995c136d926bdaf94fbe183dbf5d5aa_main ??? tmp_fd2bdf6191e61af6e801e3137e2f1102_main 6 directories, 10 files