ObjectBox Sync Connector Setup
Configure the ObjectBox Sync Server to connect to MongoDB and do an initial synchronization.
Last updated
Was this helpful?
Configure the ObjectBox Sync Server to connect to MongoDB and do an initial synchronization.
Last updated
Was this helpful?
Once you have a , you can connect the ObjectBox Sync Server to it.
Before using the MongoDB Sync Connector, let us ensure that the ObjectBox Sync Server is up and running. This involves basically two steps: having a data model and running the ObjectBox Sync Server. Read on for details.
In general, ObjectBox relies on a data model, which is typically defined as part of your (client) application. This is a different approach to MongoDB and other databases. The data model is defined by developers using special annotations in the programming language of your application. If you are not familiar with that process yet, then this is a good time to get familiar with ObjectBox "annotations" and how to define your data model. This is dependent on the programming language:
(uses a FlatBuffers schema file)
All ObjectBox build tools also generate a data model JSON file, which must be provided to the ObjectBox Sync Server. This data model is also used by the MongoDB Sync Connector to map data between ObjectBox and MongoDB (see the ).
To avoid any later issues, run and test Sync Server without connecting to MongoDB and your client application, and validate that data is synced.
See the ObjectBox Sync Server page on how to run Sync Server.
By then you should be able to reach the ObjectBox Sync Server . Navigate to the "Schema" page to see your data model, which should look similar to this:
For a visual overview, you can also try the "Type dependencies" and "Class" diagrams on the page.
Use a separate MongoDB instance for testing purposes.
Now that the Sync Server is up and running, let us connect it to MongoDB. This can be done via CLI arguments or via the configuration file.
To configure the ObjectBox MongoDB Sync Connector via CLI arguments when starting Sync Server (see ObjectBox Sync Server), you can use the following options:
--mongo-db
: The primary MongoDB database name; the "database" containing the collections used for sync. By default, this is "objectbox_sync".
Alternatively, configure the MongoDB connection in the Sync Server configuration file (see Sync Server Configuration). In your sync-server-config.json
, add a new mongoDb
node which contains key/value pairs for MongoDB specific configuration attributes:
To fully enable data synchronization, an initial import from MongoDB is necessary. This syncs all collections from MongoDB, which are part of the ObjectBox data model, into ObjectBox. It performs this with snapshot isolation level to offer a maximum level of consistency, e.g. concurrent updates from other systems do not interfere with this process (when done in accordance with MongoDB transaction semantics). From this snapshot onwards, all changes are synchronized continuously between MongoDB and ObjectBox. If one system goes offline, the synchronization will pick up where it left off, so no change will be lost.
When you navigate to a MongoDB page in the Admin UI, you will see a prominent message if the initial import has not run yet:
On the "Full Sync" page beneath the "MongoDB Connector" menu, tap the "Full Import from MongoDB" button and a dialog like this will appear:
It shows you some information and input fields. You must enter your name (required) and optional notes to help you identify this import in the future. Then tap "Import" to start the import process.
During the import you can see the progress on the page. There are two main phases:
Exporting from MongoDB: one column shows the counts of MongoDB databases, collections and documents already exported.
Importing into ObjectBox: the next column to the right shows the counts of objects actually changed in ObjectBox and the total count of all checked objects, which should roughly match the number of documents in MongoDB.
As you can see in the image above, you have the possibility to abort an ongoing import process. This is to be used very cautiously, as it can yield inconsistent states of data if importing has already started (e.g. dangling relations). Then the best course of action is to start a new full sync again and let it complete. Aborting an ongoing import should be reserved for emergencies only.
A finished import will show up as "Completed" in the "State" column on a green background. At that point, it is a good time to check the logs to ensure the sync went smoothly. Also, having a look at the "Data" page will give you a good overview of the data imported.
As seen in a previous screenshot above, the "Full Sync" MongoDB page shows a table of all current and past import processes. This history gives you a good overview of what was imported and when. These are the columns:
Started: when the import started
State: the current state of the import; it typically goes through these phases: "Started", "Exporting", "Importing", "Completed"
"Export: DBs/Coll./Docs": the counts of MongoDB databases, collections and documents exported
"Changed/Objects": the count of objects actually changed in ObjectBox and the total count of all checked objects, which should roughly match the number of documents in MongoDB
Warnings/Errors: the count of warnings and errors; details can be found in the logs
Triggered by: the name entered for this import (in the import confirmation dialog)
Note: the note entered for this import (in the import confirmation dialog)
Schema Version: The ObjectBox schema version that was current when the import started. The Admin UI has a "Schema Versions" page showing the history of past versions of the data model. Thus, you can track changes to the data model over time.
MongoDB: the URL and MongoDB version used for this import
Logs: A link to the log events. Note: a future version will filter the logs for this import.
You can also see the timeline of an import/sync process by clicking on the state. It shows when a new state was reached and should look like this:
Now that you can import an entire MongoDB database into ObjectBox, you may get huge datasets in ObjectBox that span several GB and more. It is a good idea to validate full datasets on the ObjectBox Sync Server, but there are a few caveats to consider on the client side as of now:
You need to update the ObjectBox sync clients to the new version 4.3 (or later, check release notes).
User-specific (rule-based) sync will be available in a future beta release (see roadmap). Until then all data will be synced to the clients.
We are working on these aspects with high priority and will keep you updated on progress.
--mongo-url
: The (URL or URI). This can be an empty string for the default 127.0.0.1:27017
host.
If you run into snapshot errors like "SnapshotTooOld" during an import, this is likely due to a MongoDB setting. To read the data from MongoDB, the ObjectBox Sync Connector uses the to ensure consistent reads at a single point in time (from a database perspective). MongoDB keeps snapshots for a limited time, e.g. 5 minutes by default. Thus, if reading the data from MongoDB does not complete within that time, it will fail with a snapshot history error.
This issue typically only starts with MongoDB databases containing at least 10 GB, and depending on the network and MongoDB instance speed, the limit may be much higher. If you run into this error, you may want to increase the snapshot history window by setting to a higher value. The default value is 300 (5 minutes), so adjust it according to your database size and network speed.