Reading¶

Use connect for a reader whose access terms come from its token, or Database.viewer to read your own database as someone who holds the terms you name. Every read covers only the items the reader may see, by the rule that Viewer states.

The Database methods on this page read with the operator's own token, which authorise describes under read_all.

Database¶

view ¶

view(name: str) -> Selection

The whole of one view, as a Selection that you count, sample, filter or map.

The selection reads as the operator, who sees every item.

name: the view's name, which for a view in a view group is "<group>:<key>". An unknown name is refused, and the refusal lists the views there are.

db.view("papers").count()
db.view("papers").filter({"venue": {"eq": "neurips"}}).map()

viewer ¶

viewer(terms: Sequence[str] | None = None) -> Viewer

A reader of this database that holds only the access terms given.

The reader sees only the items those terms admit, by the rule that Viewer states.

terms: the access terms, as a list or as one term in a string. Without them the reader reads with the operator's token, as db.view and the other reads on Database do. An empty list is refused because such a reader would see nothing, while a term that no item carries is accepted.

db.viewer(["cs.LG"]).view("papers").count()
db.viewer(["cs.LG"]).map()

map ¶

map(
    view: str | None = None,
    layers: Sequence[str] | None = None,
    colour_by: str | None = None,
    filters: dict | None = None,
    height: int = 480,
    size_by: str | None = None,
    basemap: Any = None,
    **kwargs,
) -> Map

An interactive map of this database for a notebook, showing every item.

This is Viewer.map read as the operator, with its arguments. To map part of a view, call db.view(name).map(), and to see what a reader holding some terms sees, call db.viewer(terms).map().

db.map(colour_by="cluster:clusters")

meta ¶

meta() -> dict

The database's structure as its server describes it, as a dictionary.

The description lists the views with their coordinate ranges, the annotation layers, the columns with the filters each one takes, and the server's limits.

item ¶

item(mosaica_id) -> dict

One item's full record, as a dictionary.

The record holds the item's fields, labels and views, as Viewer.item describes.

mosaica_id: the item's id, as a sample's mosaica_id column or a pick on the map gives it.

db.item(db.view("papers").sample(k=1).column("mosaica_id")[0].as_py())

items ¶

items(view: str, fields: Sequence[str], **options)

Every item in a view, with the fields named, as one pyarrow table.

This is Viewer.items read as the operator, who sees every item, with its keywords. With batches=True it returns the pages one at a time instead.

db.items("papers", ["title", "year"]).to_pandas()
db.items("papers", ["title"], filters={"year": {"eq": 2023}}, order="stored")

lookup ¶

lookup(
    view: str, field: str, values: Iterable[Any], fields: Sequence[str] = ()
)

Find the items in a view by their values of a unique column, as a pyarrow table.

This is Viewer.lookup read as the operator, with a row for each item whose value of field is one of values.

db.lookup("papers", "doi", ["10.1/a", "10.2/b"], fields=["title"])

artifacts ¶

artifacts(view: str, layer: str, fields: Sequence[str], **options)

Every annotation of a layer in a view, with the properties named, as one pyarrow table.

This is Viewer.artifacts read as the operator, with its keywords. With batches=True it returns the pages one at a time instead.

db.artifacts("papers", "clusters", ["key", "masked_count"])

browse_artifacts ¶

browse_artifacts(view: str, layer: str, **options) -> dict

One page of a layer's annotations, each with its mosaica_id, key, name and count.

This is Viewer.browse_artifacts as this database's own reader; options are its keywords.

ids = {one["name"]: one["mosaica_id"] for one in
       db.browse_artifacts("papers", "topics", level=0)["artifacts"]}

artifact ¶

artifact(mosaica_id: Any, view: str, **options) -> dict

One annotation's record in view.

This is Viewer.artifact as this database's own reader; options are its keywords.

db.artifact(db.browse_artifacts("papers", "topics")["artifacts"][0]["mosaica_id"],
            "papers")

viewport_artifacts ¶

viewport_artifacts(view: str, zoom: int, per_tile: int, **options)

The annotations in each map tile at one zoom level, as one pyarrow table.

This is Viewer.viewport_artifacts read as the operator, with its keywords, and per_tile limits how many annotations each level of a layer gives one tile.

db.viewport_artifacts("papers", 2, per_tile=20, layers=["clusters"])

aggregate ¶

aggregate(
    view: str,
    groupings: Sequence[dict],
    filters: dict | None = None,
    reference: dict | None = None,
    *,
    cursor: str | None = None,
) -> list

Count the items in a view, overall or by group, as one pyarrow table per grouping.

This is Viewer.aggregate read as the operator, who sees every item, with its arguments.

venue = {"count": True, "by": {"field": "venue", "top": 10}}
[size, venues] = db.aggregate("papers", [{"count": True}, venue])

categories ¶

categories(
    column: str,
    prefix: str | None = None,
    view: str | None = None,
    codes: Sequence[int] | None = None,
    filters: dict | None = None,
)

The values of a category column, as a pyarrow table.

This is Viewer.categories read as the operator, who sees every value, with its arguments.

db.categories("venue")
db.categories("venue", prefix="neur")
db.categories("venue", codes=[1, 3])

Viewer ¶

Viewer(
    url: str,
    token: Optional[TokenSource] = None,
    *,
    terms: Optional[Sequence[str]] = None,
)

The reader of one Mosaica database over HTTP, used to count, sample, read and map what its token may see.

A reader holds a set of access terms and sees an item when its terms satisfy one of the item's access labels, such as secret&(team_a|team_b). Every count, map and record it is given is computed over the items it may see. Get a reader with connect(url, token) for a database someone else runs, or with db.viewer(terms) for one of your own.

A read the server is too busy to take raises Refusal with status 429 and the wait in retry_after_s, and the reader does not retry it.

url: the address of the database's viewer plane.
token: the token to read with, in any of the forms TokenSource describes, or None to send none, as a server started with mosaica serve --open allows.
terms: the access terms the token was made for, kept for display, or None when they are not known.

token ¶

token() -> Optional[Token]

Return the token this reader sends, or None for a reader made without one.

A token close to expiry is renewed first, as TokenSource describes.

view ¶

view(name: str) -> Selection

Return the whole of one view as a Selection to narrow, count, sample, read or map.

A view is one layout of the items, with its own coordinates. name is as meta() lists it, "<group>:<key>" for a view in a view group. An unknown name is refused with a list of those this reader can see.

v.view("papers").count()

map ¶

map(
    view: Optional[str] = None,
    layers: Optional[Sequence[str]] = None,
    colour_by: Optional[str] = None,
    filters: Optional[dict] = None,
    height: int = 480,
    size_by: Optional[str] = None,
    basemap: Any = None,
    **kwargs: Any,
) -> Map

Open the interactive map of this database as a notebook widget, showing what this reader may see.

The parameters are those of Map, which fetches its data with this reader's token. A view, layer or column that this reader's meta() does not list is refused, and the refusal names those it does.

meta ¶

meta() -> dict

Return the structure of the database that this reader may see, as a dictionary.

It lists the views, layers and columns the other reads name, with how each column can be filtered and the server's limits.

item ¶

item(mosaica_id: Any) -> dict

Return one item's full record as a dictionary, when this reader may see the item.

mosaica_id: the item's id, from a sample, a read or a map pick.

An item this reader may not see is refused in the same way as one that does not exist. The record has these keys.

fields: the item's values by column name, with no entry where it has no value.
labels: why this reader may see the item, as the parts of its access labels that this reader satisfies.
views: the views this reader can find the item in, each with its position there in grid units.
scoped: its values of columns declared for a view group, by column and then by the group's key.

v.item(sample.column("mosaica_id")[0].as_py())

items ¶

items(
    view: str,
    fields: Sequence[str],
    *,
    position: Optional[bool] = None,
    labels: Optional[bool] = None,
    filters: Optional[dict] = None,
    keep_unmatched: Optional[bool] = None,
    count: Optional[bool] = None,
    order: Optional[str] = None,
    page_rows: Optional[int] = None,
    pages: Optional[int] = None,
    cursor: Optional[str] = None,
    compression: Optional[str] = None,
    batches: bool = False,
)

Read every item this reader may see in view, with the fields named, as one pyarrow table.

The server answers a read a page at a time, and items requests responses until no row remains and joins their pages. With batches=True it returns a Batches instead, to take the pages one at a time. Each option is sent only when given, and the server's own setting applies otherwise.

view: the view to read, as meta() names it. An item with no position in it is not returned.
fields: the declared columns to return, in this order. A column declared for a view group, read under a view outside it, is named "<column>@<key>".
position: True adds mosaica:x and mosaica:y, each item's position in the view's coordinates, in degrees on a geographic view.
labels: True adds mosaica:labels, the parts of each item's labels that this reader satisfies, as item gives them.
filters: a filter expression, as Selection.filter takes one, that the items returned must match.
keep_unmatched: True returns every item, with a mosaica:matched column saying whether it matches filters.
count: True puts in the head the number of items this reader may see in the view, as visible, and of those that match, as matched. It cannot be combined with cursor.
order: "map" returns the items by their place on the map and "stored" in the order the server stores them, the faster order for a field that is neither rendered nor indexed. Without it the server chooses.
page_rows: the rows in a page, capped at the server's max_page_rows.
pages: the most pages in one response.
cursor: the next of an earlier read or the cursor of a PartialRead, to read the rows after it.
compression: "zstd" compresses the pages on their way from the server.

The table holds mosaica_id, as its decimal string, and then the columns asked for, with each category column holding its values' keys. A read that finds no item returns a table of no rows with those columns. Its schema metadata mosaica.head is the head that Batches.head describes.

A response cut short, or a later request refused, raises a PartialRead.

papers = db.items("papers", ["title", "year"], position=True)
papers.to_pandas()
recent = db.items("papers", ["title"], filters={"year": {"range": {"gte": 2020}}})

lookup ¶

lookup(
    view: str, field: str, values: Sequence[Any], fields: Sequence[str] = ()
)

Find the items in view that hold values in the unique column field, returning them as a pyarrow table with mosaica_id, field and the columns in fields.

A value with no row is held by no item this reader may see.

v.lookup("papers", "doi", ["10.1/a", "10.2/b"], fields=["title"])

artifacts ¶

artifacts(
    view: str,
    layer: str,
    fields: Sequence[str],
    *,
    level: Optional[int] = None,
    parent: Any = None,
    q: Optional[str] = None,
    ids: Optional[Sequence[Any]] = None,
    filters: Optional[dict] = None,
    keep_unmatched: Optional[bool] = None,
    count: Optional[bool] = None,
    page_rows: Optional[int] = None,
    pages: Optional[int] = None,
    cursor: Optional[str] = None,
    compression: Optional[str] = None,
    palette_size: Optional[int] = None,
    batches: bool = False,
)

Read every annotation of layer that this reader may see, with the properties named, as one pyarrow table.

An annotation is one member of an annotation layer, such as a cluster, a region or a node in a taxonomy. The read is paged and resumed as an items read is. Its mosaica_id, parents and target columns hold decimal strings.

view: the view whose items the counts and geometry are computed over.
layer: the layer to read, as meta() lists it.
fields: the properties to return, in order, from "key", "level", "parents", "target" (the annotation it is attached to), "masked_count" (its items this reader may see), "content", "centroid", "box", "shape" and "slot". "centroid" gives the columns centroid_x and centroid_y, and "box" gives box_x_min, box_y_min, box_x_max and box_y_max, all in the view's coordinates.
level: return only the annotations at this level.
parent: return only the children of this annotation, given by its mosaica_id.
q: return only the annotations whose key or first text contains this, ignoring case. It cannot be combined with parent.
ids: return only the annotations these mosaica_ids name, such as the values of a sample's membership:<layer> column. An id that names nothing, or an annotation this reader is not served, gets no row.
filters: return only the annotations holding a matching item, with their matched_count.
keep_unmatched: with filters, return every annotation, with a matched_count of 0 where nothing matches.
count: put in the head served, the annotations the request selects before filters, and matched, those of them that match. It cannot be combined with cursor.
page_rows, pages, cursor, compression, batches: as for items.
palette_size: how many colours your palette holds, to fill slot as viewport_artifacts describes.

The rows come in order of level, and within a level in the order the annotations were published.

clusters = db.artifacts("papers", "clusters", ["key", "masked_count", "centroid"])

viewport_artifacts ¶

viewport_artifacts(
    view: str,
    zoom: int,
    per_tile: int,
    *,
    tiles: Optional[Sequence[int]] = None,
    bbox: Optional[Sequence[float]] = None,
    layers: Any = None,
    levels: Any = None,
    computed: Optional[Sequence[str]] = None,
    filters: Optional[dict] = None,
    highlight: Optional[dict] = None,
    budget: Optional[int] = None,
    palette_size: Optional[int] = None,
    pin: Any = None,
    on_tile: Optional[Callable[[Any], None]] = None,
)

Read the annotations a map draws in each tile at zoom, as one pyarrow table with a row per annotation per tile.

Each level of a layer gives a tile at most per_tile of the annotations holding an item this reader may see there, the largest first. An annotation's counts and geometry cover all of its items this reader may see, whichever tile holds it.

view: the view, as meta() names it.
zoom, tiles, levels: as Selection.sample takes them.
per_tile: at most the server's max_artifacts_per_tile.
bbox: read the tiles touching this box, (min_x, min_y, max_x, max_y) in the view's coordinates, in place of tiles. Without either, every tile of the view is read.
layers: the layers to read, as a list of names or "all". Without it no layer is read.
computed: the subset of "centroid" and "box" to compute. It can narrow what each layer declares and cannot widen it.
filters, highlight: filter expressions that set only each row's matched and highlighted. They do not change which rows come back.
budget: as artifact_budget on Selection.sample. A nested or dag layer is cut to it once for all the tiles read, so its rows have a null tile.
palette_size: how many colours your palette holds, from 2 to 32. Each row's slot is then the index of its colour, chosen for this reader so that neighbouring annotations differ, and an annotation keeps its slot at any zoom, box or filter.
pin: as on Selection.sample.
on_tile: a function called with each frame of rows as a pyarrow table as it arrives. The nested and dag layers' frame, whose tile is null, comes first.

Each option is sent only when given, and the server's own setting applies otherwise.

The columns are layer, mosaica_id, key, masked_count, centroid_x, centroid_y, box_min_x, box_min_y, box_max_x, box_max_y, content, parent_ids, rung (the level), matched, highlighted, target, tile and slot. The centroid and box are on the grid of 2^32 steps per axis that a sample's code uses, and mosaica_id, parent_ids and target hold decimal strings. The schema metadata holds mosaica.pin, mosaica.stale and mosaica.region as a Sample holds them.

v.viewport_artifacts("papers", 2, per_tile=20, layers=["topics"]).to_pandas()

aggregate ¶

aggregate(
    view: str,
    groupings: Sequence[dict],
    filters: Optional[dict] = None,
    reference: Optional[dict] = None,
    *,
    cursor: Optional[str] = None,
) -> list

Count how the items this reader may see in view are distributed, returning one pyarrow table per grouping in the order of groupings.

view: the view the counts are taken in. An item with no position in it is not counted.
groupings: one dictionary per table, built from the keys described below.
filters: the set counted, as a filter expression.
reference: a second set to compare with, as a filter expression, where {} is every item this reader may see in the view. Each table that counts then also has reference_count and lift.
cursor: the cursor of a PartialRead from the same request, to read the rows after it. The result then starts at the table the read stopped in, so when the stop came inside a table, the first table returned holds only its remaining rows, which you append to the last table in the error's rows.

A grouping names at least one figure. "count": True counts the items in each row, so {"count": True} alone is the size of the set. "values", "missing", "min", "max" and "mean" each take a list of number or timestamp fields and give a column per field, such as min_year, and "summary" gives all five. They are exact and are taken over every item this reader may see in the view, so they are refused beside "by" or "cells" and in a request with filters or reference. min_ and max_ have the field's type, and mean_ is a float, or a UTC timestamp on a timestamp field.

"by" groups the rows in one of three ways.

By a category field, as {"field": name, "top": n} for the n values with the most items, or {"field": name, "values": [key, ...]} for the values named. A rest row then counts the items with any other value and a none row those with no value.
By the annotations of a layer, as {"layer": name, "top": n} or {"layer": name, "artifacts": [mosaica_id, ...]}, with rest and none rows as for a field. A layer with several levels requires "level": k, which a layer with one level refuses. A listed id that names nothing, or an annotation this reader is not served, gets no row. Because annotations can overlap, the counts can sum to more than the set. "palette_size": n adds each annotation's palette slot. On a nested or dag layer, top needs "cut": {"zoom": z, "bbox": [x0, y0, x1, y1], "budget": b}, so that it ranks the annotations a map draws at that zoom, box and cluster budget, and without budget it ranks the finest cut.
By bins of a number or timestamp field, as {"field": name, "bins": n}, a histogram of at most n bins in one page. With "range": [lower, upper] the bins divide that range into equal widths and rest counts the values outside it. Without a range the edges are round values around the smallest and largest value this reader may see in the view, so they stay put as the filters change. The edges have the field's type, except that an integer field gets float edges when a bound of range is fractional. "sample": s counts a large set faster, over about s of its items, and scales each count to the set's size.

"cells": {"depth": d} divides the set, or each group, into the view's cells at a depth from 0 to 32, where up to 16 they are the map's tiles at that zoom. Adding "area": [x0, y0, x1, y1] keeps only the cells meeting that box. A histogram takes no cells.

A table's columns come in this order, each where it applies: group (listed, rest or none), key (a category's key, or an annotation's mosaica_id as its decimal string, which a member_of filter takes as it is), title, slot, lower and upper (a bin's edges, the last bin including its upper), cell (the cell's Morton prefix), count, reference_count, lift ((count / total) / (reference_count / reference_total)), and then the field figures. A category field has title only when its vocabulary holds titles. slot is null without palette_size and on the rest and none rows.

Each table's schema metadata mosaica.head holds its totals as JSON, such as total, the items in the set, and with "sample" the object sample, whose sampled says whether any count was scaled. mosaica.recomposed is "true" when the pages counted a database that changed during the read, and mosaica.region says whether a region filter was answered exactly or over the grid cells covering it.

A read that stops part of the way raises a PartialRead, whose rows are the tables read before the stop and whose cursor, passed back as cursor, reads the rest.

venue = {"count": True, "by": {"field": "venue", "top": 10}}
[size, venues] = v.aggregate("papers", [{"count": True}, venue])
cells = {"count": True, "cells": {"depth": 6}}
v.aggregate("papers", [cells], filters={"year": {"eq": 2023}}, reference={})
years = {"count": True, "by": {"field": "submitted", "bins": 20}}
[years] = v.aggregate("papers", [years])
[year] = v.aggregate("papers", [{"count": True, "summary": ["year"]}])
cut = {"zoom": 3, "bbox": [0, 0, 100, 100], "budget": 1000}
topics = {"count": True, "by": {"layer": "topics", "top": 10, "cut": cut}}
[topics] = v.aggregate("papers", [topics])

categories ¶

categories(
    column: str,
    prefix: Optional[str] = None,
    view: Optional[str] = None,
    codes: Optional[Sequence[int]] = None,
    filters: Optional[dict] = None,
)

List the values of a category column that this reader may see, as a pyarrow table with one row per value.

A category column stores a small integer code for each value. Each row gives a value's code, its key as inserted, and its display title, null where none was given.

column: the category column's name.
prefix: list only the values whose key or title, or a word in either, starts with this text, ignoring case. The rows then come in the order of the matched text, at most the server's max_suggestions of them, each with a count of the items this reader may see that carry the value. The schema metadata mosaica.more is "true" when more matched, and mosaica.total is the number of items counted.
view: the view to read the column in, needed for a column declared for a view group. With prefix it also limits each count to the items in this view.
codes: list only these codes, such as codes read from a viewport response over HTTP. It cannot be combined with prefix.
filters: a filter expression that limits each count, and needs prefix and view. A value the filter excludes keeps its row with a count of 0.

Without prefix the rows are in key order.

v.categories("venue")
v.categories("venue", prefix="neur")
v.categories("venue", codes=[3, 7])
recent = {"year": {"range": {"gte": 2020}}}
v.categories("venue", prefix="", view="papers", filters=recent)

browse_artifacts ¶

browse_artifacts(
    view: str,
    layer: str,
    level: Optional[int] = None,
    parent: Any = None,
    q: Optional[str] = None,
    filters: Optional[dict] = None,
    limit: Optional[int] = None,
    cursor: Optional[str] = None,
    palette_size: Optional[int] = None,
) -> dict

Return one page of a layer's annotations as a dictionary, independent of any map tile, for a list or search panel.

The page holds the rows in artifacts and the cursor for the next page in next, absent on the last page.

view, layer, level: as for artifacts.
parent: list the children of this annotation, given by its mosaica_id. Without it or q, the page lists the top-level annotations.
q: list the annotations whose key or name contains this text, ignoring case. It cannot be combined with parent.
filters: a filter expression. Each row then has matched_count, the number of its items this reader may see that match, and the page is ordered by it.
limit: the most rows on the page, capped at the server's max_browse_rows.
cursor: the next of the previous page.
palette_size: how many colours your palette holds, as on artifacts.

Each row has masked_count, the number of its items this reader may see, and child_count, how many rows parent= would list for it. A row has a name when the annotation has text or a label attached to it, such as a cluster's topic.

page = v.browse_artifacts("papers", "clusters")
more = v.browse_artifacts("papers", "clusters", cursor=page["next"])

artifact ¶

artifact(mosaica_id: Any, view: str, zoom: Optional[int] = None) -> dict

Return one annotation's record as a dictionary, when this reader may see the annotation.

mosaica_id: the annotation's id, as browse_artifacts, artifacts or a sample's membership:<layer> column gives it.
view: the view whose items its count and geometry are computed over.
zoom: the zoom the caller draws at, so that the shape drops vertices that would move an edge by less than a pixel. Without it the shape is simplified to at most 2,048 vertices, and a hull drawn around the items is never simplified.

The record always has layer, masked_count and rung, its level. It has key, content, centroid, box and shape where the annotation has them, with the geometry on the grid of 2^32 steps per axis that a sample's code uses.

v.artifact(page["artifacts"][0]["mosaica_id"], "papers")

Selection ¶

A part of one view, narrowed by filters and a box, that you count, sample, read or map as one reader sees it.

db.view(name) and viewer.view(name) give a whole view. Calling filter or within gives a new, narrower selection and leaves the original unchanged, so one selection can be the start of several.

papers = db.view("papers")
recent = papers.filter({"year": {"range": {"gte": 2020}}})
corner = recent.within((0, 0, 10, 10))
corner.count()

view `property` ¶

view: str

The name of the view this selection is part of.

filters `property` ¶

filters: Optional[dict]

The filters added so far, joined into one expression, or None when there are none.

box `property` ¶

box: Optional[tuple]

The box this selection is limited to, as (min_x, min_y, max_x, max_y), or None.

After several calls to within it is the overlap of their boxes, and None, matching no item, when they do not overlap.

filter ¶

filter(expression: dict) -> 'Selection'

Narrow this selection to the items that also match expression, returning a new selection.

expression is a filter written as a dictionary that maps a column name to a test, or that applies all_of, any_of or none_of to a list of expressions. An item must match every filter added.

A test on a timestamp column takes a datetime, a date, a pandas Timestamp or a NumPy datetime64, and the client sends it as microseconds since the Unix epoch. A moment with no time zone is read as UTC, and a date stands for its midnight in UTC. A number or a string, such as an RFC 3339 date, is sent as it is.

db.view("papers").filter({"year": {"range": {"gte": 2020, "lt": 2024}}})
db.view("papers").filter({"submitted": {"range": {"gte": datetime.date(2020, 1, 1)}}})

within ¶

within(box: Sequence[float]) -> 'Selection'

Narrow this selection to the items whose position lies inside box, returning a new selection.

box is (min_x, min_y, max_x, max_y) in the view's own coordinates, the ones the rows were inserted with. An item on the edge is inside, and a box whose minimum is above its maximum is refused.

db.view("papers").within((0.0, 0.0, 100.0, 50.0)).count()

count ¶

count() -> int

Count the items in this selection that its reader may see.

When the box's outline crosses more grid cells than the server's max_region_cells setting allows, the server counts the cells that cover the box, so the count can include items outside it. sample(k=0) gives the same count as matched in its mosaica.counts, with the verdict on the box in mosaica.region.

db.view("papers").count()

sample ¶

sample(
    zoom: int = 0,
    k: Optional[int] = None,
    *,
    tiles: Optional[Sequence[int]] = None,
    highlight: Optional[dict] = None,
    layers: Any = None,
    levels: Any = None,
    artifact_budget: Optional[int] = None,
    point_rows: Union[str, Sequence[str], None] = None,
    underlay_offset: Optional[int] = None,
    pin: Any = None,
    on_counts: Optional[Callable[[Any, Any], None]] = None,
) -> Sample

Sample the points a map of this selection draws at zoom, returning them as a Sample.

The server thins dense areas so that each map tile carries at most k points.

zoom: the map's zoom level from 0, the whole view as one tile, to 16.
k: the most points drawn in one tile. It defaults to the server's k_max_marks and cannot exceed its max_k.
tiles: the tiles to sample, each by its number in Z order at this zoom, in place of the box.
highlight: a second filter that adds the highlighted column without changing which points are drawn.
layers: the annotation layers, such as clusters, whose annotations tag the points, as a list of names or "all".
levels: which levels of a levelled layer tag the points, as a list of numbers or "all". Without it each layer uses the levels it declares for this zoom.
artifact_budget: the most annotations a nested or dag layer tags points with, met by using parents in place of their children.
point_rows: the rendered columns each point carries, as a list, where [] is the position alone. "full", the default, is every rendered column, and "highlight" returns only mosaica_id and highlighted to update the highlight on points you already hold.
underlay_offset: also count the items this reader may see in the tiles this many levels finer than zoom, ignoring the filters, and return those counts as sub_cells.
pin: the decoded mosaica.pin of an earlier answer, whose version this answer's mosaica.stale is compared with. It changes none of the points or counts.
on_counts: a function called with the per-tile counts and sub_cells as pyarrow tables as soon as they arrive, before the points.

Each option is sent only when given, and the server's own setting applies otherwise.

sample = db.view("papers").sample(zoom=3, k=256)
sample.to_pandas()  # needs pandas installed

items ¶

items(fields: Sequence[str], **options)

Read every item in this selection, with the fields named, as one pyarrow table.

The selection's filters and box are sent as the read's filters, so the rows are the items count() counts. options are the other keywords of Viewer.items. filters is refused because the selection supplies its own.

db.view("papers").filter({"year": {"eq": 2023}}).items(["title"]).to_pandas()

aggregate ¶

aggregate(
    groupings: Sequence[dict],
    reference: Optional[dict] = None,
    *,
    cursor: Optional[str] = None,
) -> list

Count how the items in this selection are distributed, as one pyarrow table per grouping.

The selection's filters and box are sent as the request's filters, so each table counts the items count() counts. groupings, reference and cursor are as Viewer.aggregate takes them, and a field figure such as "min" is refused on a selection with a filter or a box.

papers = db.view("papers").filter({"year": {"range": {"gte": 2020}}})
[size, venues] = papers.aggregate(
    [{"count": True}, {"count": True, "by": {"field": "venue", "top": 10}}]
)

map ¶

map(
    colour_by: Optional[str] = None,
    layers: Optional[Sequence[str]] = None,
    height: int = 480,
    size_by: Optional[str] = None,
    artifacts_per_tile: Optional[int] = None,
    budget: int = 0,
    budget_min: int = 1000,
    budget_max: int = 2000000,
    cluster_budget: int = 1000,
    cluster_budget_min: int = 10,
    cluster_budget_max: int = 10000,
    palette: Optional[str] = None,
    value_colours: Optional[dict] = None,
    cluster_colours: Optional[dict] = None,
    basemap: Any = None,
    **kwargs: Any,
) -> Map

Open the interactive map of this selection as a notebook widget.

The map opens on this view with the selection's filters applied, framed on the box when there is one and otherwise on where the items are. The box only frames the camera, so items outside it are drawn when they are in frame.

The parameters are those of Map, apart from view, filters and bbox, which the selection gives. A layer or column this reader cannot see is refused, as on Viewer.map.

db.view("papers").filter({"year": {"range": {"gte": 2020}}}).map(colour_by="venue")

Sample ¶

The points a map draws at one zoom, as Selection.sample returns them: a pyarrow table thinned for drawing, with the counts it was drawn from in its metadata.

A Sample behaves as its table, points, and has these columns.

mosaica_id: the item's id, as its decimal string.
code: the point's position as a 64-bit Morton code over a grid of 2^32 steps per axis across the view. It is absent under point_rows="highlight".
Each rendered column that point_rows asks for. A category column holds each value's key, or null for a value this reader may not see.
highlighted: whether the point matches the highlight, when one was given.
membership:<layer>: for each layer named in layers that tags a point, the mosaica_id of the finest annotation of that layer holding the point, as its decimal string, or null.

The schema metadata mosaica.counts holds the counts the sample was drawn from: visible, the items this reader may see in the tiles touched, and of those matched, highlighted and served, the rows in this table.

The metadata also says which version of the data the sample was drawn from, so that you can tell whether it has moved on. mosaica.pin is that version as JSON, which you pass back decoded as pin on a later sample or viewport_artifacts, and that later answer's mosaica.stale is then "true" when the version has changed since. The version moves at every flush, merge and compaction, so "true" says that what this reader sees may have changed. After a flush, a reader the view has already served keeps the previous version until the server's refresh reaches its session, so the flush's new items appear in its answers only then, while deletions and suppressions apply at once.

When the selection has a box or a region filter, mosaica.region says whether it was answered exactly, as "exact", or over the grid cells covering it, as "cover; depth=<d>".

sub_cells is the table of finer counts that underlay_offset asks for, or None.

Batches ¶

The pages of a read from items or artifacts with batches=True, which you iterate to get each page as a pyarrow.RecordBatch.

Use it for a read too large to hold as one table, or one you may stop early, because each response is requested only when the pages before it have been taken. Because each page's category columns hold only the keys that page uses, read_all() gives each category column one dictionary when it joins the pages.

head: the first response's head as a dictionary. It holds page_rows, the page size used, and on an items read order, the order used, along with the counts when count was asked for.
next: the cursor to pass as cursor to read on after the last page taken.
done: True once the server has said that no row remains.

for batch in db.items("papers", ["title"], page_rows=10_000, batches=True):
    frame = batch.to_pandas()

read_all ¶

read_all()

Join the pages not yet taken into one pyarrow.Table.

The table's schema metadata mosaica.head is head as JSON.

to_pandas ¶

to_pandas(**options)

Join the pages not yet taken into one pandas DataFrame, passing options to pyarrow's Table.to_pandas.

close ¶

close() -> None

Stop the read and free its place on the server, as dropping it does.

PartialRead ¶

Bases: Refusal

The error raised when a read from items, artifacts or aggregate stops part of the way, because a response was cut short or a later request was refused.

rows: the rows read before the stop, as a pyarrow table, or None when no page had arrived. From aggregate it is a list of tables, and from batches=True it is None.
cursor: the cursor to pass as cursor to the method that raised the error, which then reads the rest without repeating a row, or None when the read has to start again.
done: True when every row had arrived before the stop.
why: what stopped the read, which the message begins with.
status and retry_after_s: the refusal's status and wait, as on Refusal, when a refused request stopped the read, and None otherwise.

Reading¶

Database¶

view ¶

viewer ¶

map ¶

meta ¶

item ¶

items ¶

lookup ¶

artifacts ¶

browse_artifacts ¶

artifact ¶

viewport_artifacts ¶

aggregate ¶

categories ¶

Viewer ¶

token ¶

view ¶

map ¶

meta ¶

item ¶

items ¶

lookup ¶

artifacts ¶

viewport_artifacts ¶

aggregate ¶

categories ¶

browse_artifacts ¶

artifact ¶

Selection ¶

view property ¶

filters property ¶

box property ¶

filter ¶

within ¶

count ¶

sample ¶

items ¶

aggregate ¶

map ¶

Sample ¶

Batches ¶

read_all ¶

to_pandas ¶

close ¶

PartialRead ¶

view `property` ¶

filters `property` ¶

box `property` ¶