Skip to content

Commit a2c11ce

Browse files
waxlampwaxlamp
authored
Merge pull request #2155 from dandi/atpath-design
Design document for "atpath" endpoint
2 parents 328f808 + 01de272 commit a2c11ce

File tree

1 file changed

+222
-0
lines changed

1 file changed

+222
-0
lines changed

‎doc/design/atpath.md‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
Endpoint for Efficiently Querying an Asset or Folder at a Given Path
2+
====================================================================
3+
4+
[`dandidav`](https://github.com/dandi/dandidav), the WebDAV interface to the
5+
DANDI Archive, fetches all of its information about the Archive via the Archive
6+
REST API; unfortunately, the API's support for getting all of the necessary
7+
information about assets & folders at a given path is incredibly suboptimal,
8+
resulting in multiple API requests per WebDAV request, including making a
9+
separate request for each & every asset immediately within a folder.
10+
11+
This design doc therefore proposes the addition of a new endpoint to the
12+
Archive API that can be used to answer all of the following questions in a
13+
single (possibly paginated) request:
14+
15+
- Is the resource at a given path an asset, a folder, or nonexistent?
16+
17+
- If the resource is an asset, what are its properties and, optionally,
18+
its metadata?
19+
20+
- Optionally, if the resource is a folder, what are all of its children — both
21+
assets and subfolders — and, for child assets, what are their properties and
22+
metadata?
23+
24+
Requests
25+
--------
26+
27+
A `GET` endpoint shall be added at
28+
`/dandisets/{dandiset_id}/versions/{version_id}/assets/atpath/`. The endpoint
29+
shall take the following query parameters:
30+
31+
- `path` — A normalized forward-slash-separated relative path, identifying the
32+
asset or folder to return information on.
33+
34+
- If this parameter is absent or empty, the request is for the root
35+
directory of the Dandiset version's file hierarchy.
36+
37+
- If the path is present but no resource exists at the given path, a 404
38+
response is returned.
39+
40+
- For the purposes of this endpoint, empty folders do not exist.
41+
42+
- If the path ends in a forward slash and no folder exists at the given
43+
path, a 404 is returned, even if an asset exists at the path with the
44+
trailing slash removed.
45+
46+
- `metadata` — Whether to include asset metadata in the response; possible
47+
values are `false` (the default), indicating that metadata should not be
48+
included, and `true`, indicating that metadata should be included.
49+
50+
- Values that cannot be parsed to `true` or `false` using Django's
51+
`BooleanField` result in a 400 response.
52+
53+
- `children` — Whether to include children of `path` in the response; possible
54+
values are `false` (the default), indicating that children should not be
55+
included, and `true`, indicating that children should be included.
56+
57+
- This parameter has no effect when the resource at `path` is an asset
58+
rather than a folder.
59+
60+
- Values that cannot be parsed to `true` or `false` using Django's
61+
`BooleanField` result in a 400 response.
62+
63+
- `page` and `page_size` — Pagination parameters
64+
65+
Responses
66+
---------
67+
68+
All successful responses from this endpoint shall be structured as a paginated
69+
list, using the same pagination schema as is already used for other endpoints.
70+
71+
- When `path` points to an asset, the list shall contain a single item: a JSON
72+
object containing the following fields:
73+
74+
- `"type"` — Always has a value of `"asset"`
75+
76+
- `"resource"` — A sub-object containing the properties of the asset at
77+
`path` (i.e., `asset_id`, `blob`, `zarr`, `path`, `size`, `created`, and
78+
`modified`). If `metadata=1` was supplied in the request, this
79+
sub-object shall also contain a `"metadata"` field containing the asset's
80+
metadata.
81+
82+
- When `path` points to a non-root folder, the first element of the list shall
83+
be a JSON object containing the following fields:
84+
85+
- `"type"` — Always has a value of `"folder"`
86+
87+
- `"resource"` — A sub-object containing the following fields:
88+
89+
- `"path"` — the path of the folder, without a trailing slash
90+
91+
- `"total_assets"` — an integer giving the total number of assets in
92+
this folder and all of its descendants
93+
94+
- `"total_size"` — an integer giving the total size in bytes of all
95+
assets in this folder and all of its descendants
96+
97+
If the `children` parameter was not set to `true` in the request, this is the
98+
only element of the list. Otherwise, the rest of the list consists of JSON
99+
objects for the assets and subfolders immediately within the folder, ordered
100+
by their paths (without trailing slashes) and sorted in UTF-8 byte-wise
101+
order.
102+
103+
- Each asset is represented by an object with the same schema as described
104+
above for when `path` points to an asset, including `"metadata"` being
105+
present when `metadata=1` was supplied in the request.
106+
107+
- Each subfolder is represented by an object with the same schema as
108+
described above for the folder at `path`.
109+
110+
- When `path` is empty or absent, the list shall be the same as for a folder
111+
located at the root of the file hierarchy, except that there shall be no
112+
initial element describing the resource at `path`. (Hence, if `children` is
113+
not `true`, the list shall be empty.)
114+
115+
Example
116+
-------
117+
118+
A request for
119+
`/dandisets/000029/versions/0.231017.2004/assets/atpath/?metadata=true&children=true`
120+
would have a response of:
121+
122+
```json
123+
{
124+
"count": 9,
125+
"next": null,
126+
"previous": null,
127+
"results": [
128+
{
129+
"type": "asset",
130+
"resource": {
131+
"asset_id": "1ab0d4ff-1231-48c9-a8a3-8145552830a4",
132+
"blob": "2fd7464f-5459-4c96-a938-27cf13f4d330",
133+
"zarr": null,
134+
"path": "bar-renamed-3334",
135+
"size": 4,
136+
"created": "2023-09-27T17:08:32.039960Z",
137+
"modified": "2023-10-17T19:55:40.394757Z",
138+
"metadata": "--- SNIPPED FOR BREVITY ---"
139+
}
140+
},
141+
{
142+
"type": "asset",
143+
"resource": {
144+
"asset_id": "ad49cd65-184a-4dad-8dde-523e7aaab56c",
145+
"blob": "56f5b879-e5fa-476c-9893-dab482f66b3d",
146+
"zarr": null,
147+
"path": "baz",
148+
"size": 4,
149+
"created": "2023-03-16T15:50:06.908811Z",
150+
"modified": "2023-03-17T15:41:18.403892Z",
151+
"metadata": "--- SNIPPED FOR BREVITY ---"
152+
}
153+
},
154+
{
155+
"type": "folder",
156+
"resource": {
157+
"path": "sub-anm369962",
158+
"total_assets": 1,
159+
"total_size": 6644036
160+
}
161+
},
162+
{
163+
"type": "folder",
164+
"resource": {
165+
"path": "sub-anm369963",
166+
"total_assets": 1,
167+
"total_size": 6393196
168+
}
169+
},
170+
{
171+
"type": "folder",
172+
"resource": {
173+
"path": "sub-anm369964",
174+
"total_assets": 1,
175+
"total_size": 7660100
176+
}
177+
},
178+
{
179+
"type": "folder",
180+
"resource": {
181+
"path": "sub-monk-g",
182+
"total_assets": 1,
183+
"total_size": 18295752
184+
}
185+
},
186+
{
187+
"type": "folder",
188+
"resource": {
189+
"path": "sub-RAT123",
190+
"total_assets": 1,
191+
"total_size": 18792
192+
}
193+
},
194+
{
195+
"type": "asset",
196+
"resource": {
197+
"asset_id": "c7913fad-ccc6-467f-a4b6-a9b62af6d98f",
198+
"blob": "0317cf5a-4047-4e19-aae1-4f7b7434d2d7",
199+
"zarr": null,
200+
"path": "test1234",
201+
"size": 9,
202+
"created": "2023-03-16T15:50:06.762183Z",
203+
"modified": "2023-03-17T15:41:18.369626Z",
204+
"metadata": "--- SNIPPED FOR BREVITY ---"
205+
}
206+
},
207+
{
208+
"type": "asset",
209+
"resource": {
210+
"asset_id": "bb612483-0a97-4309-9763-7fb518265c70",
211+
"blob": "0317cf5a-4047-4e19-aae1-4f7b7434d2d7",
212+
"zarr": null,
213+
"path": "test12345",
214+
"size": 9,
215+
"created": "2023-03-16T15:50:07.184996Z",
216+
"modified": "2023-03-17T15:41:18.378517Z",
217+
"metadata": "--- SNIPPED FOR BREVITY ---"
218+
}
219+
}
220+
]
221+
}
222+
```

0 commit comments

Comments
 (0)