Release v2.0.0

These release notes list changes compared to version 1.5.2 (previous stable).

For more on each item, please consult the
API Reference
and the Documentation.

New features

API Tables

Added Table support in the clients

• New Database methods: .get_table(), .create_table(), .list_tables(), .list_table_names(), .drop_table(), .drop_table_index().
• New classes: Table, AsyncTable (with notable methods .definition(), .create_index(), .create_vector_index(), .list_index_names(), .alter() in addition to those similar as for Collections).
• New classes TableFindCursor and AsyncTableFindCursor for find results iteration.
• New classes TableInsertOneResult, TableInsertManyResult, TableInsertManyException for table-specific results and exceptions.
• New classes to create/inspect tables: TableInfo, CreateTableDefinition, ListTableDescriptor, ListTableDefinition, TablePrimaryKeyDescriptor, TableValuedColumnType, TableKeyValuedColumnType, TableVectorColumnType, TableUnsupportedColumnType, TableColumnTypeDescriptor, TableScalarColumnTypeDescriptor, TableVectorColumnTypeDescriptor, TableValuedColumnTypeDescriptor, TableKeyValuedColumnTypeDescriptor, TableUnsupportedColumnTypeDescriptor, TableAPISupportDescriptor.
• New classes to create/inspect table indexes: TableIndexType, TableIndexOptions, TableVectorIndexOptions, TableBaseIndexDefinition, TableIndexDefinition, TableVectorIndexDefinition, TableAPIIndexSupportDescriptor, TableUnsupportedIndexDefinition, TableIndexDescriptor.
• New classes to alter tables: AlterTableOperation, AlterTableAddColumns, AlterTableDropColumns, AlterTableAddVectorize, AlterTableDropVectorize.

Maps in tables can be expressed as list of pairs (association lists):

• Support for automatic handling of DataAPIMaps (+possibly dicts) in the proper table payload portions
• Introduced serdes option encode_maps_as_lists_in_tables (default to "NEVER") to control this

Tables come with full typing support (usage is optional), much like is now for Collections.

Find-and-rerank for Collections

Added support for findAndRerank ("hybrid search") API command

Method .find_and_rerank() in Collection/AsyncCollection classes:

• New cursor classes [Async]CollectionFindAndRerankCursor (returned by said method).
• Introduced class RerankedResult, over which these cursors iterate (modulo custom mappings).

Support for reranker header-based authentication where needed:

• New authentication classes RerankingHeadersProvider, RerankingAPIKeyHeaderProvider for collections.
• Introduced reranking_api_key parameter for APIOptions, also exposed in:{get|create}_{table|collection} database methods; collection/table with_options; collection/table to_[a]sync methods.

Changes to .create_collection() method of [Async]Database:

• New classes CollectionLexicalOptions, CollectionRerankOptions, RerankServiceOptions.
• New lexical and rerank entries in CollectionDefinition (and associated builder interface management).

New [async_]find_reranking_providers methods in database admin class:

• New class hierarchy introduced: RerankingProviderParameter, RerankingProviderModel, RerankingProviderToken, RerankingProviderAuthentication, RerankingProvider, FindRerankingProvidersResult to express the response.

Other new features

Introduced new astrapy-specific data types to fully express database values (see serdes_options below for fine control):

• DataAPIVector data type.
• DataAPIDate, DataAPITime, DataAPITimestamp, DataAPIDuration data types.
  • (e.g. instead of 'datetime.datetime', instances of DataAPITimestamp are returned by default.)
• DataAPISet, DataAPIMap collective data types.

Introduced a consistent API Options system:

• An APIOptions object supplied at DataAPIClient() creation and inherited at each "spawn" operation, with overrides.
• Sensible defaults if nothing supplied.
• serdes_options to control data types accepted for writes and to select data types for reads.
  • Defaults prefer custom data types for lossless expressivity (set serdes_options.custom_datatypes_in_reading = False to fall back to stdlib types).
  • serdes_options.binary_encode_vectors, to control usage of binary-encoding for writing vectors (on by default for Collections).
  • Numbers are treated by default as ints and floats. Set serdes_option.use_decimals_in_collections = True to switch to Decimal.
  • Fine control over naive-datetime tolerance and timezone is introduced (see accept_naive_datetimes and
    datetime_tzinfo options). Acceptance of naive datetimes is off by default.
• Support for arbitrary 'database' and 'admin' headers throughout the object chain.
• Fully reworked timeout options through all abstractions:
  • TimeoutOptions has six classes of timeouts, applying differently to various methods according to the kind of method. Timeouts can be overridden on a per-method-call basis
  • Removal of the 'max_time_msparameter (keeping a quick migration path of replacing it withtimeout_ms` throughout).
  • Timeout of 0 means completely disabled.

Typing support for Collections (optional):

• Database .get_collection() and .create_collection() methods have a document_type parameter, for spawning instances matching a type hint of Collection[MyType].
• If unspecified, fallback to DefaultCollection = Collection[dict[str, Any]].
• Cursors from find also support strict typechecking.
• Async collections mirror this typing behaviour.

Arbitrary field names and dot-escaping:

• Utilities astrapy.utils.document_paths.escape_field_names and astrapy.utils.document_paths.unescape_field_path.
• The distinct method can accept a list of (literal) str|int as well as whole (escaped) path-identifier strings.

Support for Astra DB "custom domain" endpoints for database (in which case: .id, .region, .get_database_admin(), .info() and .name() aren't available.)

Changes and improvements

Collections and Cursors

DataAPIMap, DataAPIDate accepted for writes. For dates, the same timezone caveats as for datetime.date apply.

Cursors' API changes (breaking changes):

• Cursors are typed, similarly to Tables and Collections. The find method has an optional document_type parameter for typechecking.
• Introduced .map() and .to_list() methods.
• .state() now has values in the CursorState enum (for a string, use my_cursor.state.value).
• '.address' is removed.
• .rewind() returns None, mutates cursor in-place.
• '.distinct()' removed: use the corresponding collection(/table) method.
• '.keyspace' property removed.
• '.retrieved' removed: use .consumed.
• Added several cursor management methods (see docstrings for details).
• The .clone() method does not strip the mapping anymore, rather retains it.
• Removed the 'alive' property (replace with the check cursors.state != CursorState.CLOSED).
• Class hierarchy around cursors revised:
  • Renamed the abstract root-class 'FindCursor' to AbstractCursor (accommodating "find and rerank" cursors).
  • Cursor classes renamed to [Async]CollectionFindCursor (likewise for table cursors).

Creation of collections with .create_collection() database method changed substantially:

• A single definition: CollectionDefinition parameter with the whole structure, instead of a growing list of "flat" parameters.
  • This parameter can also be a plain dictionary if desired.
  • Replaces and removes 'dimension', 'metric', 'source_model', 'service', 'indexing', 'default_id_type' (all of them subsumed in definition).
• CollectionDefinition: implemented fluent interface to build collection definition objects.
• Removed parameters 'additional_options' and 'timeout_ms' from .create_collection() (the latter as part of the broader timeout rework).
• This is also the return type of Collection.options() (formerly called 'CollectionOptions').
• The database .list_collections() method returns a list of CollectionDescriptor items, one of whose attributes is the CollectionDefinition as used in creation.

Exceptions

Exception hierarchy revised in a breaking way:

Revised API and semantics for the "bulk operation" exceptions:

• The four CollectionInsertManyException, CollectionUpdateManyException, CollectionDeleteManyException, TableInsertManyException classes now inherit directly from DataAPIResponseException.
• New structure for [Collection|Table]InsertManyException: they have members inserted_ids(/inserted_id_tuples) and an exceptions list for the root cause(s).
• New structure for Collection[Update|Delete]ManyException: they have members partial_result and a single-exception cause. They are now raised consistently for API exceptions occurring during the respective methods.

Introduced InvalidEnvironmentException for operations invalid on some Data API environments.

Removed several exceptions not needed anymore:

• removed 'CursorIsStartedException': now CursorException raised for all state-related illegal calls in cursors.
• removed 'CollectionNotFoundException', replaced by a ValueError in the few cases it's needed.
• removed 'CollectionAlreadyExistsException' class (not used anymore once 'check_exists' parameter removed from create_collection).
• removed 'DataAPIDetailedErrorDescriptor'.
• removed 'CumulativeOperationException`.

Renamed some exceptions:

• renamed 'InsertManyException' to CollectionInsertManyException.
• renamed 'DeleteManyException' to CollectionDeleteManyException.
• renamed 'UpdateManyException' to CollectionUpdateManyException.
• renamed 'DevOpsAPIFaultyResponseException' to UnexpectedDevOpsAPIResponseException.
• renamed 'DataAPIFaultyResponseException' to UnexpectedDataAPIResponseException.

String representation improved for: DataAPIResponseException, DataAPIDetailedErrorDescriptor.

All astrapy exceptions derive directly from Exception (and not 'ValueError' anymore).

Replaced the ValueError not directly coming from function calls/constructors with more appropriate exceptions.

Admin- (and schema-) related changes

Spawner methods for databases/admins do not issue DevOps API calls anymore and have standardized their signature and behaviour:

• AstraDBAdmin.get_[async]_database():
  • Does not run DevOps API calls anymore (for missing keyspace/region).
  • Defers defaults to the [Async]Database class consistently.
  • Removed 'database_admin_timeout_ms', 'request_timeout_ms', 'timeout_ms' parameters.
  • region now required if id passed instead of endpoint.
• AstraDBDatabaseAdmin.get[_async]_database():
  • removed 'database_admin_timeout_ms', 'request_timeout_ms', 'timeout_ms' parameters.
• AstraDBDatabaseAdmin.get_database_admin() standardized and simplified:
  • does not run DevOps API calls anymore (for missing keyspace/region).
  • removed 'database_admin_timeout_ms', 'request_timeout_ms', 'timeout_ms' parameters;
  • region now required if id passed instead of endpoint.
• removed 'normalize_region_for_id' utility method, not used anymore.

Signature change from {"ok": 1} to None (void methods) for some admin and schema methods:

• AstraDBAdmin: drop_database (and its async counterpart).
• AstraDBDatabaseAdmin, DataAPIDatabaseAdmin: create_keyspace, drop_keyspace, drop (+ async).
• Database, AsyncDatabase: drop_collection, drop_table.
• Collection, AsyncCollection: drop.

removed the 'CommandCursor'/'AsyncCommandCursor' classes:

• AstraDBAdmin: list_databases, async_list_databases methods return regular lists.
• [Async]Database: list_collections, list_tables methods return regular lists.

[Async]Database: added a .region property.

Database info structure: changed class name and reworked attributes of AstraDBAdminDatabaseInfo (formerly 'AdminDatabaseInfo') and AstraDBDatabaseInfo (formerly 'DatabaseInfo').

Switching to the api_options parameter consistently brings many changes to the classes' with_options and to_[a]sync methods:

• DataAPIClient
  • _copy(), with_options(): removed 'callers'.
  • get_..._database...(): removed 'api_path', 'api_version'.
  • get_admin(): removed 'dev_ops_url', 'dev_ops_api_version'.
• AstraDBAdmin
  • _copy: removed 'environment', 'dev_ops_url', 'dev_ops_api_version', 'callers'
  • with_options: removed 'callers'.
  • create..._database: added token, spawn_api_options.
  • get..._database: removed 'api_path', 'api_version', 'database_request_timeout_ms', 'database_timeout_ms'; renamed 'database_api_options' to spawn_api_options.
  • get_database_admin: added token, spawn_api_options.
• AstraDBDatabaseAdmin
  • _copy(): removed 'api_endpoint', 'environment', 'dev_ops_url', 'dev_ops_api_version', 'api_path', 'api_version', 'callers'.
  • with_options(): removed 'api_endpoint', 'callers'.
  • get..._database(): removed 'api_path', 'api_version', 'database_request_timeout_ms', 'database_timeout_ms'; renamed 'database_api_options' to spawn_api_options.
• DataAPIDatabaseAdmin
  • _copy(): removed 'api_endpoint', 'environment', 'api_path', 'api_version', 'callers'.
  • with_options(): removed 'api_endpoint', 'callers'.
  • get..._database(): removed 'api_path', 'api_version', 'database_request_timeout_ms', 'database_timeout_ms'; renamed 'database_api_options' to spawn_api_options.
• [Async]Database
  • _copy(): removed 'api_endpoint', 'callers', 'environment', 'api_path', 'api_version'.
  • with_options(): removed 'callers'; added token.
  • to_[a]sync(): removed 'api_endpoint', 'callers', 'environment', 'api_path', 'api_version'.
  • get_collection(): removed 'collection_request_timeout_ms', 'collection_timeout_ms'; renamed 'collection_api_options' to spawn_api_options.
  • get_table(): removed 'table_request_timeout_ms', 'table_timeout_ms'; renamed 'table_api_options' to spawn_api_options.
  • create_collection(): removed 'collection_request_timeout_ms', 'collection_timeout_ms'; renamed 'collection_api_options' to spawn_api_options.
  • create_table(): removed 'table_request_timeout_ms', 'table_timeout_ms'; renamed 'table_api_options' to spawn_api_options.
  • get_database_admin(): removed 'dev_ops_url', 'dev_ops_api_version'.
• [Async]Collection
  • _copy(): removed 'request_timeout_ms', 'collection_timeout_ms', 'callers'.
  • with_options: removed 'request_timeout_ms', 'collection_timeout_ms', 'name', 'callers'.
  • to_[a]sync(): removed 'request_timeout_ms', 'collection_timeout_ms', 'keyspace', 'name', 'callers'.
• [Async]Table
  • _copy(): removed 'database', 'name', 'keyspace', 'request_timeout_ms', 'table_timeout_ms', 'callers'.
  • with_options: removed 'name', 'request_timeout_ms', 'table_timeout_ms', 'callers'.
  • to_[a]sync(): removed 'database', 'name', 'keyspace', 'request_timeout_ms', 'table_timeout_ms', 'callers'.

Other changes (some breaking)

Some (naming) changes prompted by the need to generalize to Tables/Collections:

• Renamed 'CollectionVectorServiceOptions' class to VectorServiceOptions.
• Renamed 'astrapy.constants.SortDocuments' to SortMode.
• Renamed (collection-specific) "result" classes as follows:
  • 'DeleteResult' to CollectionDeleteResult.
  • 'InsertOneResult' to CollectionInsertOneResult.
  • 'InsertManyResult' to CollectionInsertManyResult.
  • 'UpdateResult' to CollectionUpdateResult.

[Async]Database.drop_collection method does not accept an object anymore, only the string name of the target to drop.

[Async]Database drop_collection method now accepts a keyspace parameter.

AsyncDatabase methods get_collection and get_table are not async functions anymore (remove the 'await' when calling them).

The following "info" methods are made async (i.e. awaitable): AsyncDatabase.info(), AsyncDatabase.name(), AsyncCollection.info(), AsyncTable.info(), AsyncDatabase.list_collections(), AsyncDatabase.list_tables()

[Async]Collection and [Async]Database: .info() method now accepts the relevant timeout parameters.

Improved StrEnum matching for e.g. better coercion of TableIndexType (and future enum with e.g. dashes in values).

• Renamed parameter 'collection_name' to collection_or_table_name and allow for keyspace=None in database command() method.

Removal of deprecated modules, objects, components, parameters, interfaces

Version 2 of the clients marks the removal of several long-deprecated items:

• 'core' (i.e. pre-1.0) library.
• 'collection.bulk_write' and the associated result and exception classes.
• 'vector=', 'vectorize=' and 'vectors=' parameters from collection methods.
• 'set_caller' method of DataAPIClient, AstraDBAdmin, DataAPIDatabaseAdmin, AstraDBDatabaseAdmin, [Async]Database, [Async]Collection.
• 'caller_name' and 'caller_version' parameters. A single list-of-pairs callers is now expected.
• 'id' and 'region' to DataAPIClient's 'get_database' (and async version). Use api_endpoint which is now the one positional parameter.
  • Accordingly, the syntax client[api_endpoint] also does not accept a database ID anymore.
• 'region' parameter of AstraDBDatabaseAdmin.get[_async]_database (was ignored already in the method).
• 'namespace' parameter of several methods of: DataAPIClient, admin objects, Database and Collection (use keyspace).
• 'namespace' property of CollectionInfo, DatabaseInfo, CollectionNotFoundException, CollectionAlreadyExistsException (use keyspace).
• 'namespace' property of Database and Collection (switch to keyspace).
• 'update_db_namespace' parameter for keyspace admin methods (use update_db_keyspace).
  -'use_namespace' for Databases (switch to use_keyspace).
  -'delete_all' method of Collection and AsyncCollection (use delete_many({})).

Other secondary removed items:

• 'check_exists' parameter removed from [Async]Database.create_collection method: the client does no checks now.
• static method from_api_endpoint of AstraDBDatabaseAdmin (reason: unused).
• 'database' parameter to the to_sync() and to_async() conversion methods for collections.

Internal changes, repo maintenance/management

• Restored support for Python 3.8, 3.9 (certaing type variables were not compatible with the syntax).
• API payloads are encoded with full Unicode (not encoded in ASCII anymore) for HTTP requests.
• Switch to DSE6.9 for local non-Astra testing.
• Full restructuring of tests and CI (tables+collections on same footing; and other related improvements).
• Adopt blockbuster in async tests to detect (and bust) any blocking call.
• Classes in the hierarchy other than DataAPIClient have breaking changes in their constructor (now options-first and keyword-arg-only). Not user-facing as the user is never supposed to directly instantiate any of them.
• Token and Embedding API key coercion into *Provider now happens at the Options' init layer.
• [Async]Collection.find_one method uses the actual findOne API command.
• Rename main branch from 'master' to main.
• Major restructuring of the codebase in directories (some internal-only imports changed; reduced the scope of test_imports).
• Collection and Table insert_many methods employ the Data API returnDocumentResponses primitive under the hood.
• Removal of unused imports from toplevel __init__.py (ids, constants, cursors).
• Simplified timeout management classes and representations.