MongoDB Sync Connector

Bi-directional Data Sync with MongoDB - on-premise and to the cloud

ObjectBox Data Sync syncs data with MongoDB using the integrated MongoDB Sync Connector. Changes made on ObjectBox clients are synchronized in real-time to MongoDB and vice versa.

Get your MongoDB Sync Connector Alpha

Bi-directional Synchronization with MongoDB

ObjectBox Sync Connector for MongoDB: Architecture

ObjectBox Sync brings your data in MongoDB to the edge (e.g. mobile and IoT devices, big and small servers) and synchronizes changes back to MongoDB. By using ObjectBox Sync, you can make your MongoDB data always available: continue to work offline and sync in real-time when online.

Roadmap

If you are coming from Atlas Device Sync, you already now that it reaches its end-of-life on September 30, 2025. Thus, it's a good idea to contact ObjectBox asap to get started with your migration.

While ObjectBox Sync is an established product, the MongoDB Connector is in its early stages. You can still get started with POCs or integrations right away. The timeline below enables you to align your development with ours and pick up speed. Once new features are out, you profit immediately.

Planned Date	Released	Connector Version	Main features
Oktober 2024	✅	Alpha 1	Two-way sync between MongoDB and ObjectBox for changes that are happening live.
January 2025	✅	Alpha 2	JWT authentication, Mapping for all MongoDB data types
February 2025	✅	Alpha 3	Relation mapping (to-one and many-to-many)
March 2025	✅	Alpha 4	Support for additional MongoDB ID types
April 2025	✅	Alpha 5 (2025-04-01)	Use MongoDB transactions, better change tracking, improved error handling, improved MongoDB Admin page
April 2025		Beta 1	Import initial data from MongoDB, pick-up sync at any point
April/May 2025		Beta 2	Server-side rules to select data for sync per user
Mai 2025		1.0	Initial release incorporating most important user feedback

We'll update the plan regularly.

Setup

Do not connect to your MongoDB production instance! Use a separate instance for testing purposes.

Setting up Sync server and its MongoDB Sync Connector involves the following steps:

1. Create and provide a Data Model to Sync Server using a model JSON file.
2. Run and test Sync Server without connecting to MongoDB and validate it actually syncs data.
3. Ensure that your MongoDB instance is a replica set. This is required for the MongoDB Sync Connector to work.
4. Configure the MongoDB connection and run Sync Server. E.g. provide the connection URL to the Sync Server and restart it. This can be done via CLI arguments or the JSON configuration file.
5. Verify the MongoDB connection using the Admin UI.

Read on for details.

Create and Provide a Data Model to Sync Server

In general, the ObjectBox Sync server requires a data model to be provided (a JSON file, see ObjectBox Sync Server). This data model is also used by the MongoDB Sync Connector to map data between ObjectBox and MongoDB. On how this works, see the chapter on data mapping below.

Run and test Sync Server

To avoid any later issues, run and test Sync Server without connecting to MongoDB and your client application and validate data is synced.

See the ObjectBox Sync Server page on how to run Sync Server.

Ensure that your MongoDB instance is a Replica Set

MongoDB Atlas clusters are already replica sets, no additional configuration is required.

Only a MongoDB replica set instance provides the necessary features for the MongoDB Sync Connector to work (e.g. MongoDB's change streams).

A local standalone MongoDB instance (MongoDB Community Edition is fine) can be converted to a replica set. You can do this either by following the official MongoDB documentation, or by following these simplified steps (tested on Ubuntu Linux) for a single node setup:

1. Stop the MongoDB service: sudo systemctl stop mongod

2. Edit the MongoDB configuration file: sudo vi /etc/mongod.conf

3. Add the following lines to the configuration file:

   replication:
     replSetName: "rs0"

4. Start the MongoDB service: sudo systemctl start mongod

5. Connect to the MongoDB shell: mongosh

6. Initialize the replica set via the MongoDB shell: rs.initiate()

Configure the MongoDB connection and run Sync Server

To configure the ObjectBox MongoDB Sync Connector via CLI arguments when starting Sync Server (see ObjectBox Sync Server), you can use the following options:

• --mongo-url: The MongoDB connection string (URL or URI). This can be an empty string for the default 127.0.0.1:27017 host.
• --mongo-db: The primary MongoDB database name; the "database" containing the collections used for sync. By default this is "objectbox_sync".

If you are using Docker on Windows/macOS to run an instance of the ObjectBox Sync server, use host.docker.internal as the host in the MongoDB connection string for the --mongo-url parameter, for example,

docker run --rm -it \
    --volume "$(pwd):/data" \
    --user $UID \
    --publish 127.0.0.1:9999:9999 \
    --publish 127.0.0.1:9980:9980 \
    objectboxio/sync:sync-server-${sync_server_version} \
    --model /data/objectbox-model.json \
    --unsecured-no-authentication \
    --admin-bind 0.0.0.0:9980 \
    --mongo-url mongodb://host.docker.internal:27017 \
    --mongo-db test-db

It enables the Sync server running within the container to access the MongoDB instance running on the host system. Note, it only works on Windows and macOS.

Alternatively, configure the MongoDB connection in the Sync Server configuration file (see Sync Server Configuration). In your sync-server-config.json, add a new mongoDb node which contains key/value pairs for MongoDB specific configuration attributes:

{
    ...
    "mongoDb": {
        "url": "1.2.3.4",
        "database": "db123"
    }
}

If your Sync server version's date is lower than 2024-10-07, use {"mongoUrl": "1.2.3.4"} without the mongoDb config node.

Verify the MongoDB connection

Use the ObjectBox Sync Server Admin web app to verify the MongoDB connection works.

Syncing and mapping data with MongoDB

Difference in terminology and concepts

ObjectBox and MongoDB have many similarities. Nevertheless, it's important to understand the differences in terminology and concepts between the two databases. The following table illustrates these differences. It serves as background information on how to map things between the two systems:

Concept	ObjectBox	MongoDB
Database containing the data	A store, grouped into types (data classes).	A database, grouped into collections.
What your application "opens"	Using a name or directory a single store (database) is opened locally on device. (In the case of Sync, this is the "client database".) Multiple stores can be opened, which are strictly separate.	A client is used to connect to a MongoDB server. All databases can be accessed on the server remotely.
Arranging data in a database (e.g. data sets)	A box holds all objects of the same type. A type typically matches a data class in programming languages. It's part of a strict schema (data model) that is enforced. The type definition consists of a fixed set of properties.	Collections are used to group documents together. By default, no strict rules are imposed (schema-less).
Data record	An object is an instance of a type. It can have data values for the properties defined in the type.	A document is a set of data values called fields. It's very similar to a JSON file.
Modelling related data	Objects can have to-one and to-many relationships to other objects. Relationships are bidirectional. Data is typically normalized.	Typically, a document embeds all "related data" into itself resulting in larger documents. Data is typically not normalized. Alternatively, one can also choose to use references, which is more similar to relationships.
Many-to-many relationships	Fully supported. Unlike to-one relationships, the data is stored outside the object. Updating relations is very efficient. There's no user-defined order and no duplicates.	Alternative modelling by embedding or referencing documents. Inside the document, you can have an array of IDs in a user-defined order allowing duplicates.
Nested data records	Nested data is supported by flex properties. E.g. these allow maps, which can contain nested data structures (JSON-like).	Documents contain nested data by default.
Identifiers (IDs)	IDs are 64-bit integers. They are unique within its box and database instance.	MongoDB uses Object ID (short: OID; 12 bytes) by default and supports other ID types. OIDs are unique within their collection and likely globally unique.

Mapping data from MongoDB

The data model used by ObjectBox defines types, which are mapped to MongoDB collections. Similarly, the properties of a type are mapped to keys inside a MongoDB document. Thus, you should ensure that the ObjectBox data model matches the MongoDB schema. E.g. if you have an existing MongoDB database, ensure to match the names when you create the ObjectBox model.

Objects and Documents

When you compare the data structure of simple data elements, the difference is not big. Without nested data, the main difference you will note is the ID type.

Note: nested documents are supported via the ObjectBox "Flex" property type, which can hold a map-like (JSON-like) structure. We are also considering alternatives to this, so let us know if you have specific requirements.

ID mapping

ObjectBox Sync automatically maps IDs when syncing with an external system like MongoDB. This way, you can use the native IDs in each system, 64-bit integer IDs in ObjectBox and e.g. 12-byte object IDs in MongoID. This also means that MongoDB ID is not present in ObjectBox objects and vice versa.

ObjectBox IDs are only valid on their local device. Do not store them manually (apart from relations) e.g. as a custom list or vector, when you want to sync to other devices. For details, you can refer to the internal ID mapping docs that happens on each ObjectBox device.

Besides the Object ID, ObjectBox supports most common ID types offered by MongoDB. IDs of incoming documents from MongoDB are automatically detected and mapped to ObjectBox local IDs. This mapping is persisted and thus any change made on the ObjectBox side can be mapped back to the initial ID type and value.

For newly created (inserted) objects on the ObjectBox side, a new MongoDB object ID (OID) is created by default. You can customize the MongoDB ID types in the ObjectBox data model: for the ID property, define a "external property type" on the ID property. E.g. a definition of the ID field in the ObjectBox entities would look similar like this pseudo code (syntax varies according to programming language):

@Id @ExternalType(UUID) int64 id;

The following table shows the supported ID types:

MongoDB type	Incoming from MongoDB	IDs for new documents created in ObjectBox
Object ID	✅	✅ This is the default type
UUID (Binary with UUID subtype)	✅	✅ External types: Uuid (V7) or UuidV4
String	✅	✅ External types: UuidString (V7) or UuidV4String
Binary	✅	Uses default MongoDB Object Id
Int64	✅	Uses default MongoDB Object Id
Int32	✅	Uses default MongoDB Object Id

To-One Relations

If you want to learn more about ObjectBox relations, check the relation documentation.

ObjectBox Sync also happens automatically maps IDs used in relations (starting with version Alpha 3 of MongoDB Sync Connector).

Consider ObjectBox to-one relations, which have a single relation property pointing to another object using a 64-bit integer ID. This becomes a reference field in MongoDB's document containing the MongoDB object ID (OID). See the following illustration for an example:

To-One Relationship Example: motherId

The supported ID types also apply for relations, e.g. a "Person" document would have an UUID as the _id field value, relations to it would also use the UUID as the relation ID on the MongoDB side.

Many-to-Many Relations

Many-to-many relations work a bit different. As illustrated in the table above, many-to-many relations work differently in ObjectBox and MongoDB. For mapping between the following rules apply:

• On the MongoDB side, many-to-many relations are stored as an array of ID references:

  • The IDs are stored inside the document "owning" the relationship.

  • The owning side of a relationship is always the same type (collection).

  • If you want to, you can make this relation bidirectional, by adding IDs to the "target side" of the relationship. Just do not make this visible in the ObjectBox data model.

• On the ObjectBox side, its native many-to-many relationships are used:

  • They are bidirectional, e.g. you can define it on the owning and target side.

  • They can be updated efficiently without touching the object

  • They can be used in queries to link types (aka join).

As to-many relations consist of ID values, all supported types can be used. In theory, different ID types can be used in the same to-many, relation. However, it's usually good practice to stick to a single ID type per MongoDB collection if possible.