MscmpSystDb (mscmp_syst_db v0.1.0)
A database management Component for developing and managing database-per-tenant
oriented systems. To achieve this we wrap and extend the popular Ecto and
EctoSql libraries with a specialized templated (EEx) migrations system and add
additional, opinionated abstractions encapsulating the tenant model as it
relates to development, data access, and runtime concerns.
Important
"Database-per-tenant" is not the typical tenancy implementation pattern for Elixir/Phoenix based applications. As with most choices in software architecture and engineering there are trade-offs between the different tenancy approaches that you should be well-versed with prior to committing to this or any other tenancy model for your applications.
Concepts
There are several concepts requiring definitions which should be understood before continuing. Most of these concepts relate to runtime concerns though understanding them will inform your sense of the possibilities and constraints on development and deployment scenarios.
Datastore
A Datastore can most simply be thought of as a single database created to support either a tenant environment or an administrative function of the application. More specifically speaking, a Datastore establishes a store of data and a security boundary at the database level for the data of a tenant or of administrative functionality.
Using MscmpSystDb.create_datastore/2 automatically will create the database
backing the Datastore.
Datastores and the Ecto dynamic repositories which back them are started and stopped at runtime using this Component's API. Datastores are not typically started directly via OTP application related functionality at application startup. This is chiefly because we don't assume to even know what Datastores actually exist until we've started up an administrative Datastore which records the information.
Datastore Context
A Datastore Context represents a PostgreSQL database role which is used to establish Datastore access and security contexts using database security features. Datastore Contexts are specific to a single Datastore and are managed by the this Component, including the creation, maintenance, and dropping of them as needed, typically in conjunction with Datastore creation/deletion.
Behind the scenes Datastore Contexts use the "Ecto Dynamic Repositories" feature. Each Datastore Context is backed by an Ecto Dynamic Repo. Starting a Datastore Context starts its Ecto Dynamic Repo including establishing the connections to the database. Stopping a Datastore Context shuts that associated Dynamic Repo down and terminates its database connections.
There are several different kinds of Datastore Contexts which can be defined:
Owner: This kind of Datastore Context creates a database role to serve as the database owner of all the database objects backing the Datastore making it the de facto admin role for the Datastore. While the Owner Datastore Context owns the database objects backing the Datastore, it is only a regular database role (no special database rights) and it cannot be a database login role itself. All Datastores must have exactly one Owner Datastore Context defined.
Login: The Login Datastore Context is a regular database role with which the application can log into the database and perform operations allowed by the database security policies established by the database developer. There can be one or more Login Datastore Contexts in order to support various security profiles that the application may assume or in order to build connection pools with varying limits depending on some application specific need (e.g. connections support web user interface vs. connections supporting external API interactions.). For a Datastore to be useful there must be at least one Login Datastore Context defined for the Datastore.
Non-Login: While the Owner Datastore Context is required, there are other possible scenarios where non-login roles could be useful in managing access to specific database objects, but how useful Non-Login roles might be will depend on application specific factors; the expectation is that their use will be rare. Naturally, there is no requirement for Non-Login Datastore Contexts to be defined for any Datastore.
Finally, when we access the database from the application we'll always be doing
so identifying one of our Login Datastore Contexts. This is done using
MscmpSystDb.put_datastore_context/1 which behind the scenes is using the
Ecto.Repo dynamic repository features (Ecto.Repo.put_dynamic_repo/1).
Note that there is no default Ecto Repo, dynamic or otherwise, defined in the
system. Any attempts to access a Datastore Context without having established
the current Datastore Context for the process will cause the process to crash.
Warning!
Datastore Contexts are created and destroyed by the application using the API functions in this Component. The current implementation of Login Datastore Contexts, however, is expected to have certain security weaknesses related to database role credential management.
With this in mind, do not look to our implementation as an example of how to approach such a problem until this and other warnings disappear. The reality is that while in certain on-premises scenarios our current approach might well be workable, it was designed with the idea of kicking the can of a difficult and sensitive problem down the road and not as a final solution that we'd stand behind. We do believe this problem is solvable with sufficient time and expertise.
Database Development
Our development model assumes that there are fundamentally two phases of development related to the database: Initial/Major Development and Ongoing Maintenance.
Initial/Major Development
When initially developing a database schema, prior to any releases of usable software the typical "migrations" oriented development pattern of a continuing sequence of incremental changes is significantly less useful than it is during later, maintenance oriented phases of development. During initial development it is more useful to see database schema changes through the lens of traditional source control methodologies. The extend to which this is true will naturally vary depending on the application. Larger, database-centric applications will benefit from this phase of development significantly more than smaller applications where the database is simple persistence and data isn't significant beyond this persistence support role.
Ongoing Maintenance
Once there is an active release of the software and future deployments will be
focused on maintaining already running databases, our model shifts to the norms
typical of the traditional migrations database development model. We expect
smaller, relatively independent changes which are simply applied in sequence.
Unlike other migration tools such as the migrator built into EctoSql, we have
some additional ceremony related to sequencing migrations, but aside from these
minor differences our migrations will resemble those of other tools once in the
maintenance phase of development.
Note
Despite the discussion above, the distinction between "Initial/Major
Development" and "Ongoing Maintenance" is a distinction in developer practice
only; the tool itself doesn't make this distinction but merely is designed to
work in a way which supports a workflow recognizing these phases. The cost of
being able to support the Initial/Major Development concept is that migrations
are not numbered or sequenced automatically as will be shown below. If you
don't need the Initial/Major Development phase, the traditional EctoSql
migrator may be more suitable to your needs.
Source Files & Building Migrations Overview
In the more typical migrations model, the migration files are themselves the source code of the database changes. This Component separates the two concepts:
Database source code files are written by the developer as the developer sees fit. Database source files are what we are most concerned with from a source control perspective; and these files can be freely modified and changes committed up to the point that they are built into released migrations. Database source files are written in (mostly) plain SQL;
EExtags are allowed in the SQL and can be bound during migration time.Once the database source code has reached some stage of completion, the developer can use the
mix builddbtask to generate migration files from the database sources. In order to build the migration files, the developer will create aTOML"build plan" file which indicates which database source files to include in the migrations and their sequence. For more about the build process and build plans see themix builddbtask documentation.
Now let's connect this back to the development phases discussed previously.
During the "Initial/Major Development" phase, we expect that there will be many
database source files and that these files will be written, committed to source
control, modified, and re-committed to source control not as migrations but as
you would any other source file (for example, maybe one file per table.); we
might also be building migration files at this time for testing purposes, but
until the application is released we'd expect the migration files to be cleaned
out and rebuilt. Finally once tests, code, reviews, etc. are complete and a
release is ready to be prepared, a final mix builddb is run to create the
release migrations and those migrations are committed to source control.
From this point forward we generally wouldn't modify the original database
source files or the final release migrations: the release migrations are
essentially set in stone once they may be deployed to a environment where
dropping the database is not an option. Subsequent development in the "Ongoing
Maintenance" phase looks similar to traditional migration development. For any
modification to the database you'll create new a database source file(s) for
those modifications specifically and they'll get new version numbers which will
in turn create new migrations when mix builddb builds them. These will then
be deployed to the database as standard migrations would.
Migration Deployments
Once built, migration files are deployed to a database similar to the way
traditional migration systems perform their deployments: the migrations are
checked, in migration number order, against a special database table listing the
previously deployed migrations (table ms_syst_db.migrations). If a migration
has been previously deployed, it's skipped and the deployment process moves onto
the next migration; if the migration needs to be deployed it is applied to the
database and, assuming successful deployment, the process moves onto the next
migration or exits when all outstanding migrations have been applied.
Each individual migration is applied in a single database transaction. This means that if part of a migration fails to apply to the database successfully, the entire migration is rolled back and the database will be in the state of the last fully successful migration application. A migration application failure will abort the migration process, cancelling the attempted application of migrations after the failed migration.
Unlike the EctoSql based migration process, migrations in MscmpSystDb are
expected to be managed at runtime by the application. There is no external
mix oriented migration deployment process. Migration processes are started
for each tenant database individually allowing for selective application of
migrations to the specified environment or allowing for "upgrade window" style
functionality. Migrations are also EEx templates and template bindings can be
supplied to the migrator to make each deployment specific to the database being
migrated if appropriate. Naturally, much depends on the broader application
design, but the migrator can support a number of different scenarios for
deployment of database changes.
Finally, the migrator, can in a single application, manage and migrate different database schemas/migration sets depending on the identified "type". This means that different database schemas for different subsystems can be supported by the migration system in a single application. This assumes that a single database is of a single type; that type may be any of the available types, but mixing of types in a single database is not allowed.
Custom Database Types
Ecto, EctoSql, and the underlying PostgreSQL library Postgrex offer decent
PostgreSQL data type support out of box, but they don't directly map some of the
database types that can be helpful in business software such as PostgreSQL range
types, internet address types, and interval types. To this end we add some
custom database data types via the modules in the MscmpSystDb.DbTypes.*
namespace.
Data Access Interface
The Ecto library offers a data access and manipulation API via the Ecto.Repo
module. We wrap and in some cases extend the majority of that functionality in
this Component as documented in the Query section. As a rule of
thumb, you want to call on this module for such needs even if the same can be
achieved with the Ecto library. This recommendation is not meant to suggest
that you shouldn't use the Ecto.Query related DSL or methods for constructing
queries; using the Ecto Query DSL is, in fact, recommended absent compelling
reason to do otherwise.
Telemetry
Each category represents both a functional area and an implied security level based on the database connection privileges required:
Categories and Security Implications
:datastore- DBA Security Level Database and context lifecycle operations requiring DBA connection privileges. All operations in this category have high security sensitivity regardless of apparent complexity (e.g., even state checking requires DBA access).:migrator- Privileged Security Level Schema migration and versioning operations requiring privileged database access.:service- Standard Security Level Application-level service management using regular database connections.:query- Standard Security Level Query execution and transaction operations using regular database connections.:utility- No Database Connection Utility functions that don't require database access.
Security Monitoring Patterns
- High-privilege operations:
[:datastore, :migrator] - Application-level operations:
[:service, :query] - All database operations:
[:datastore, :migrator, :service, :query]
Summary
Query
A convenience function that currently wraps the Ecto.Repo.aggregate/4
function.
A convenience function that currently wraps the Ecto.Repo.all/2 function.
A convenience function that currently wraps the Ecto.Repo.delete/2
function.
A convenience function that currently wraps the Ecto.Repo.delete!/2
function.
A convenience function that currently wraps the Ecto.Repo.delete_all/2
function.
A convenience function that currently wraps the Ecto.Repo.exists?/2
function.
A convenience function that currently wraps the Ecto.Repo.get/3
function.
A convenience function that currently wraps the Ecto.Repo.get!/3
function.
A convenience function that currently wraps the Ecto.Repo.get_by/3
function.
A convenience function that currently wraps the Ecto.Repo.get_by!/3
function.
A convenience function that currently wraps the Ecto.Repo.in_transaction?/0
function.
A convenience function that currently wraps the Ecto.Repo.insert/2
function.
A convenience function that currently wraps the Ecto.Repo.insert!/2
function.
A convenience function that currently wraps the Ecto.Repo.insert_all/3
function.
A convenience function that currently wraps the Ecto.Repo.insert_or_update/2
function.
A convenience function that currently wraps the Ecto.Repo.insert_or_update!/2
function.
A convenience function that currently wraps the Ecto.Repo.load/2
function.
A convenience function that currently wraps the Ecto.Repo.one/2
function.
A convenience function that currently wraps the Ecto.Repo.one!/2
function.
A convenience function that currently wraps the Ecto.Repo.preload/3
function.
A convenience function that currently wraps the Ecto.Repo.prepare_query/3
function.
Executes a database query and returns all rows.
Executes a database query and returns all rows. Raises on error.
Executes a database query but returns no results.
Executes a database query but returns no results. Raises on error.
Executes a database query and returns a single row.
Executes a database query and returns a single row. Raises on error.
Executes a database query returning a single value.
Executes a database query returning a single value. Raises on error.
Returns the record count of the given queryable argument.
A convenience function that currently wraps the Ecto.Repo.reload/2
function.
A convenience function that currently wraps the Ecto.Repo.reload!/2
function.
A convenience function that currently wraps the Ecto.Repo.rollback/1
function.
A convenience function that currently wraps the Ecto.Repo.stream/2
function.
A convenience function that currently wraps the Ecto.Repo.transaction/2
function.
A convenience function that currently wraps the Ecto.Repo.update/2
function.
A convenience function that currently wraps the Ecto.Repo.update!/2
function.
A convenience function that currently wraps the Ecto.Repo.update_all/3
function.
Datastore Management
Creates a new Datastore along with its contexts.
Creates database roles to back all requested Datastore contexts.
Drops a Datastore along with its contexts.
Drops the requested Datastore contexts.
Returns the state of the requested Datastore contexts.
Returns the state of the Datastore and its contexts based on the provided Datastore Options.
Datastore Migrations
Returns the most recently installed database migration version number.
Updates a Datastore to the most current version of the given type of Datastore.
Runtime
Retrieves either atom name or pid/0 of the currently established Datastore
context, unless no context has been established.
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Starts database connections for all of login contexts in the Datastore options.
Starts a database connection for the specific Datastore context provided.
Disconnects the database connections for all of the login Datastore option contexts.
Disconnects the database connection for the specific Datastore context provided.
Utility
Extracts the PostgreSQL error code and message from a given exception.
Query
A convenience function that currently wraps the Ecto.Repo.aggregate/4
function.
@spec all(Ecto.Queryable.t(), Keyword.t()) :: [Ecto.Schema.t()]
A convenience function that currently wraps the Ecto.Repo.all/2 function.
A convenience function that currently wraps the Ecto.Repo.delete/2
function.
A convenience function that currently wraps the Ecto.Repo.delete!/2
function.
A convenience function that currently wraps the Ecto.Repo.delete_all/2
function.
A convenience function that currently wraps the Ecto.Repo.exists?/2
function.
A convenience function that currently wraps the Ecto.Repo.get/3
function.
A convenience function that currently wraps the Ecto.Repo.get!/3
function.
A convenience function that currently wraps the Ecto.Repo.get_by/3
function.
A convenience function that currently wraps the Ecto.Repo.get_by!/3
function.
A convenience function that currently wraps the Ecto.Repo.in_transaction?/0
function.
A convenience function that currently wraps the Ecto.Repo.insert/2
function.
A convenience function that currently wraps the Ecto.Repo.insert!/2
function.
A convenience function that currently wraps the Ecto.Repo.insert_all/3
function.
A convenience function that currently wraps the Ecto.Repo.insert_or_update/2
function.
A convenience function that currently wraps the Ecto.Repo.insert_or_update!/2
function.
A convenience function that currently wraps the Ecto.Repo.load/2
function.
A convenience function that currently wraps the Ecto.Repo.one/2
function.
A convenience function that currently wraps the Ecto.Repo.one!/2
function.
A convenience function that currently wraps the Ecto.Repo.preload/3
function.
A convenience function that currently wraps the Ecto.Repo.prepare_query/3
function.
@spec query_for_many(iodata(), [term()], Keyword.t()) :: {:ok, %{ :rows => nil | [[term()] | binary()], :num_rows => non_neg_integer(), optional(atom()) => any() }} | {:error, Mserror.DbError.t()}
Executes a database query and returns all rows.
@spec query_for_many!(iodata(), [term()], Keyword.t()) :: %{ :rows => nil | [[term()] | binary()], :num_rows => non_neg_integer(), optional(atom()) => any() }
Executes a database query and returns all rows. Raises on error.
@spec query_for_none(iodata(), [term()], Keyword.t()) :: :ok | {:error, Mserror.DbError.t()}
Executes a database query but returns no results.
Executes a database query but returns no results. Raises on error.
@spec query_for_one(iodata(), [term()], Keyword.t()) :: {:ok, [any()]} | {:error, Mserror.DbError.t()}
Executes a database query and returns a single row.
Executes a database query and returns a single row. Raises on error.
@spec query_for_value(iodata(), [term()], Keyword.t()) :: {:ok, any()} | {:error, Mserror.DbError.t()}
Executes a database query returning a single value.
Executes a database query returning a single value. Raises on error.
Returns the record count of the given queryable argument.
A convenience function that currently wraps the Ecto.Repo.reload/2
function.
A convenience function that currently wraps the Ecto.Repo.reload!/2
function.
A convenience function that currently wraps the Ecto.Repo.rollback/1
function.
A convenience function that currently wraps the Ecto.Repo.stream/2
function.
@spec transaction((-> any()) | Ecto.Multi.t(), Keyword.t()) :: {:ok, any()} | {:error, Mserror.DbError.t()}
A convenience function that currently wraps the Ecto.Repo.transaction/2
function.
A convenience function that currently wraps the Ecto.Repo.update/2
function.
A convenience function that currently wraps the Ecto.Repo.update!/2
function.
A convenience function that currently wraps the Ecto.Repo.update_all/3
function.
Datastore Management
@spec create_datastore(MscmpSystDb.Types.DatastoreOptions.t(), Keyword.t()) :: {:ok, MscmpSystDb.Types.database_state_values(), [MscmpSystDb.Types.ContextState.t()]} | {:error, Mserror.DbError.t()}
Creates a new Datastore along with its contexts.
The creation of a new Datastore includes the following steps:
- Creating database roles representing each of the Datastore contexts.
- Creating a new database to back the Datastore.
- Applying database connection privileges to the context roles.
- Initializing the Datastore with necessary structures and data.
Parameters
datastore_options- ADatastoreOptionsstruct defining the Datastore and its contexts.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:migrations_schema(String.t/0) - The database maintenance schema used to host the migrations state table. The default value is"ms_syst_db".:migrations_table(String.t/0) - The name of the table used to store database migration state data. The default value is"migrations".
Returns
{:ok, :ready, list(ContextState.t())}- if the Datastore is successfully created.{:error, Mserror.DbError.t()}- if there's an error during the creation process.
Errors
The function may return an error with code :database_error if there's a
failure in any step of the Datastore creation process.
@spec create_datastore_contexts( MscmpSystDb.Types.DatastoreOptions.t(), [MscmpSystDb.Types.DatastoreContext.t(), ...], Keyword.t() ) :: {:ok, [MscmpSystDb.Types.ContextState.t(), ...]} | {:error, Mserror.DbError.t()}
Creates database roles to back all requested Datastore contexts.
Usually Datastore contexts are created in the create_datastore/1 call, but
over the course of time it is expected that applications may define new
contexts as needs change. This function allows applications to add new
contexts to existing Datastores.
Parameters
datastore_options- The Datastore configuration struct to use for the operation.datastore_contexts- A nonempty list of Datastore contexts to create.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details..
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.
Returns
{:ok, nonempty_list(ContextState.t())}if successful, wherenonempty_list(ContextState.t())is a list ofContextStatestructs representing the state of each created context.{:error, Mserror.DbError.t()}if there is an error creating the contexts.
@spec drop_datastore(MscmpSystDb.Types.DatastoreOptions.t(), Keyword.t()) :: :ok
Drops a Datastore along with its contexts.
Dropping a Datastore will drop the database backing the Datastore from the database server as well as all of the database roles associated with the Datastore as defined by the provided database options.
Prior to dropping the Datastore, all active connections to the Datastore should be terminated, or the function call could fail.
Warning!
This is an irreversible, destructive action. Any successful call will result in permanent data loss.
Parameters
datastore_options- ADatastoreOptionsstruct defining the Datastore and its contexts.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.:bypass_stop_datastore(boolean/0) - If true, functions such asdrop_datastore/1will not attempt to stop the Datastore. This is useful in cases, such as inMix.Tasks.Dropdb, where the datastore was never started or stopped using other means. The default value isfalse.
Returns
:okif the Datastore is successfully dropped.{:error, Mserror.DbError.t()}if there's an error during the drop process.
Errors
The function may return an error with code :database_error if there's a failure
in any step of the Datastore drop process, such as being unable to drop the
database or roles due to active connections.
@spec drop_datastore_contexts( MscmpSystDb.Types.DatastoreOptions.t(), [MscmpSystDb.Types.DatastoreContext.t(), ...], Keyword.t() ) :: :ok | {:error, Mserror.DbError.t()}
Drops the requested Datastore contexts.
This function will drop the database roles from the database server that correspond to the requested Datastore contexts. You should be sure that the requested Datastore contexts do not have active database connections when calling this function as active connections are likely to result in an error condition.
Parameters
datastore_options- The Datastore configuration struct to use for the operation.datastore_contexts- A nonempty list of Datastore contexts to drop.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details..
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.:bypass_stop_datastore(boolean/0) - If true, functions such asdrop_datastore/1will not attempt to stop the Datastore. This is useful in cases, such as inMix.Tasks.Dropdb, where the datastore was never started or stopped using other means. The default value isfalse.
Returns
:okon success{:error, reason}on failure.
@spec get_datastore_context_states( MscmpSystDb.Types.DatastoreOptions.t(), Keyword.t() ) :: {:ok, [MscmpSystDb.Types.ContextState.t(), ...]} | {:error, Mserror.DbError.t()}
Returns the state of the requested Datastore contexts.
This function will check for each given context that: it exists, whether or not database connections may be started for it, and whether or not database connections have been started.
Note that only startable contexts are included in this list. If the context
is not startable or has id: nil, the context will be excluded from the
results of this function.
Parameters
datastore_options- ADatastoreOptionsstruct defining the Datastore and its contexts.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details..
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.
Returns
{:ok, list(ContextState.t())}if successful, wherelist(ContextState.t())is a list ofContextStatestructs for each context.{:error, Mserror.DbError.t()}if there's an error retrieving the context states.
Errors
The function may return an error with code :database_error if there's a
failure in retrieving the context states.
@spec get_datastore_state(MscmpSystDb.Types.DatastoreOptions.t(), Keyword.t()) :: {:ok, MscmpSystDb.Types.database_state_values(), [MscmpSystDb.Types.ContextState.t()]} | {:error, Mserror.DbError.t()}
Returns the state of the Datastore and its contexts based on the provided Datastore Options.
This function performs the following checks:
- Verifies the existence of the database backing the Datastore.
- Checks the state of each database role representing the Datastore Contexts.
- Determines if database connections for the Datastore Contexts have been started.
Parameters
datastore_options- ADatastoreOptionsstruct defining the Datastore and its contexts.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.
Returns
{:ok, database_state, context_states}- if successful, where:database_stateis the state of the Datastore database (:readyor:not_found)context_statesis a list ofContextStatestructs for each context
{:error, Mserror.DbError.t()}- if there's an error retrieving the Datastore state.
Errors
The function may return an error with code :database_error if there's a failure
in retrieving the Datastore state.
Datastore Migrations
@spec get_datastore_version(MscmpSystDb.Types.DatastoreOptions.t(), Keyword.t()) :: {:ok, String.t()} | {:error, Mserror.DbError.t()}
Returns the most recently installed database migration version number.
The version is returned as the string representation of our segmented version
number in the format RR.VV.UUU.SSSSSS.MMM where each segment represents a
Base 36 number for specific versioning purposes. The segments are defined as:
RR- The major feature release number in the decimal range of 0 - 1,295.VV- The minor feature version within the release in the decimal range of 0 - 1,295.UUU- The update patch number of the specified release/version in the decimal range of 0 - 46,655.SSSSSS- Sponsor or client number for whom the specific migration or version is being produced for in the decimal range of 0 - 2,176,782,335.MMM- Sponsor modification number in the decimal range of 0 - 46,655.
See mix builddb for further explanation version number segment meanings.
Parameters
datastore_options- The Datastore configuration struct to use for the operation.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details..
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.
Returns
{:ok, version}- The current version of the Datastore as a string.{:error, reason}- An error occurred.
@spec upgrade_datastore( MscmpSystDb.Types.DatastoreOptions.t(), String.t(), Keyword.t(), Keyword.t() ) :: {:ok, [String.t()]} | {:error, Mserror.DbError.t()}
Updates a Datastore to the most current version of the given type of Datastore.
If a Datastore is already up-to-date, this function is basically a "no-op" that returns the current version. Otherwise, database migrations for the Datastore type are applied until the Datastore is fully upgraded to the most recent schema version.
Parameters:
datastore_options- TheDatastoreOptionsstruct containing the Datastore configuration.datastore_type- A string representing the type of the Datastore which determines which migrations are applied.migration_bindings- A keyword list of bindings to be used in the migration scripts.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options:
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:migrations_schema(String.t/0) - The database maintenance schema used to host the migrations state table. The default value is"ms_syst_db".:migrations_table(String.t/0) - The name of the table used to store database migration state data. The default value is"migrations".:migrations_root_dir(String.t/0) - The directory relative to the project directory where the database migration files are located. The default value is"priv/database".
Returns:
{:ok, [String.t()]}- A tuple containing:okand a list of migration scripts that were applied.{:error, Mserror.DbError.t()}- A tuple containing:errorand anMscmpSystErrorstruct representing the error that occurred.
Runtime
Retrieves either atom name or pid/0 of the currently established Datastore
context, unless no context has been established.
Returns
atom()- The currently established Datastore Context atom name, if the Datastore Context was named and established for the session using the standard Ecto dynamic repository naming conventions.pid()- The currently established Datastore Context pid, if the current Datastore Context was set using a pid. This will be the case when string based Datastore Context names were used.nil- If no Datastore Context was established for the session.
@spec put_datastore_context( pid() | Ecto.Repo.t() | Ecto.Adapter.adapter_meta() | GenServer.name() ) :: {:ok, atom() | pid()} | {:error, Mserror.DbError.t()}
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Using this function will set the given Datastore Context in the Process Dictionary of the process from which the function call is made.
This version of the function uses normal Ecto dynamic repository naming conventions.
Parameters
context- The Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Returns
{:ok, atom() | pid()}- The previously set Datastore Context value, if one was previously set, or the newly set value if none was previously set.{:error, Mserror.DbError.t()}- If there was an error setting the Datastore Context.
@spec put_datastore_context( Msutils.Types.Process.registry(), Msutils.Types.Process.name() ) :: {:ok, atom() | pid()} | {:error, Mserror.DbError.t()}
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Using this function will set the given Datastore Context in the Process Dictionary of the process from which the function call is made.
This version of the function allows you identify a Datastore Context using a string based name registered in the provided registry. Naturally, the Datastore Context must have been started using a string based name for this method to be applicable.
Parameters
context_registry- The registry where the Datastore Context is registered.context- The name of the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Returns
{:ok, atom() | pid()}- The previously set Datastore Context value, if one was previously set, or the newly set value if none was previously set.{:error, Mserror.DbError.t()}- If there was an error setting the Datastore Context.
@spec put_datastore_context!( pid() | Ecto.Repo.t() | Ecto.Adapter.adapter_meta() | GenServer.name() ) :: atom() | pid()
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
This function works the same as put_datastore_context/1 except that it
directly returns the result of the operation or raises on error.
Parameters
context- The Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Returns
atom() | pid()- The previously set Datastore Context value, if one was previously set, or the newly set value if none was previously set.
Raises
Mserror.DbError- If there was an error setting the Datastore Context.
@spec put_datastore_context!( Msutils.Types.Process.registry(), Msutils.Types.Process.name() ) :: atom() | pid()
Establishes the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
This function works the same as put_datastore_context/2 except that it
directly returns the result of the operation or raises on error.
Parameters
context_registry- The registry where the Datastore Context is registered.context- The name of the Datastore Context to use for Datastore interactions in the Elixir process where this function is called.
Returns
atom() | pid()- The previously set Datastore Context value, if one was previously set, or the newly set value if none was previously set.
Raises
Mserror.DbError- If there was an error setting the Datastore Context.
@spec start_datastore(MscmpSystDb.Types.DatastoreOptions.t(), Keyword.t()) :: {:ok, :all_started | :some_started, [MscmpSystDb.Types.ContextState.t()]} | {:error, Mserror.DbError.t()}
Starts database connections for all of login contexts in the Datastore options.
Parameters:
datastore_options- ADatastoreOptionsstruct containing the login contexts.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options:
:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.:datastore_name(GenServer.name/0ornil) - Specifies the name for the Datastore Supervisor. If this option is not provided, thedatastore_options.datastore_namevalue will be used as the default name for the Datastore Supervisor. If this value identifies a process registry (e.g.{:via, Registry, {MyApp.Registry, :my_registry}}), this registry will become the default registry for all Datastore Contexts; a validcontext_registryvalue overrides this default.
Returns:
{:ok, :all_started | :some_started, list(Types.ContextState.t())}- Returns:okwith either:all_startedor:some_startedatom indicating if all or some of the contexts were started successfully, along with a list ofTypes.ContextStatestructs representing the state of each context.{:error, Mserror.DbError.t()}- Returns an error tuple with a MscmpSystError struct if there was an error starting the database connections.
@spec start_datastore_context( MscmpSystDb.Types.DatastoreOptions.t(), Msutils.Types.Process.name() | MscmpSystDb.Types.DatastoreContext.t(), Keyword.t() ) :: {:ok, pid()} | {:error, Mserror.DbError.t()}
Starts a database connection for the specific Datastore context provided.
Parameters:
datastore_options- Thet:MscmpSystDb.DatastoreOptions.t/0struct containing the Datastore options.context- TheMscmpSystDb.Types.context_name/0atom ort:DatastoreContext.t/0struct representing the Datastore context.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options:
:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.
Returns:
{:ok, pid()}on success, wherepid()is the process ID of the Datastore context.{:error, reason}on failure, wherereasonis at:MscmpSystDb.MscmpSystError.t/0struct.
@spec stop_datastore( MscmpSystDb.Types.DatastoreOptions.t() | [MscmpSystDb.Types.DatastoreContext.t()] | [%{context_name: Msutils.Types.Process.name()}], Keyword.t() ) :: :ok
Disconnects the database connections for all of the login Datastore option contexts.
Parameters
datastore_options_or_contexts- ADatastoreOptionsstruct, a list ofDatastoreContextstructs, or a list of maps with:context_namekeys.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.
Returns
:okif the connections were successfully stopped.
@spec stop_datastore_context( pid() | atom() | MscmpSystDb.Types.DatastoreContext.t(), Keyword.t() ) :: :ok
Disconnects the database connection for the specific Datastore context provided.
Parameters
context- The Datastore context to disconnect. This can be apid(),atom(), orDatastoreContextstruct.opts- a Keyword List of additional key/value call configurations. See the "Options" section for details.
Options
:db_shutdown_timeout(timeout/0) - The timeout in milliseconds to wait for the database to shutdown prior to raising an error. The default value is60000.:context_registry(Msutils.Types.Process.registry/0) - Specifies the registry to use for registering named Datastore Contexts. Can be:local,:global, or a tuple of{module(), term()}. Commonly, this is a tuple of{Registry, registry_name}.
Returns
:ok- On successful stopping of the requested Datastore Context.
Utility
@spec get_pg_exception(Exception.t()) :: MscmpSystDb.Types.error_code() | Exception.t()
Extracts the PostgreSQL error code and message from a given exception.
If the PostgreSQL SQLSTATE of the exception is one of our application custom
error codes, we'll map the SQLSTATE code to an appropriate atom representation
of the error code. Our custom error codes are documented in the return type
t:Types.error_code/0.
Returns
Returns either a tuple as defined by t:Types.error_code/0 or returns the
original exception if it is not a Postgrex.Error of either a standard
PostgreSQL SQLSTATE code or a custom error code defined by the our
application.
Examples
An example of parsing our application's known custom error codes:
iex> MscmpSystDb.get_pg_exception(
...> %Postgrex.Error{
...> postgres: %{pg_code: "PM003", message: "An Example Error"}
...> })
{:msdata_syst_defined, "An Example Error"}An example of parsing a standard PostgreSQL SQLSTATE code (mocked here):
iex> MscmpSystDb.get_pg_exception(
...> %Postgrex.Error{
...> message: "Elixir Error Text",
...> postgres: %{
...> pg_code: "23502",
...> code: :not_null_violation,
...> message: "PostgreSQL Error Text"
...> }
...> })
{:not_null_violation, "Elixir Error Text"}An example of returning the original exception if it is not otherwise handled:
iex> MscmpSystDb.get_pg_exception(%ArgumentError{message: "Elixir Error Text"})
%ArgumentError{message: "Elixir Error Text"}