ObjectBox Sync enabled library

The standard ObjectBox (database) library does not include an ObjectBox Sync implementation. Depending on the programming language, it will include the Sync API, but not the implementation. For example, ObjectBox Java in its standard version allows compiling using the Sync API, it won't any Sync logic due to the missing implementation.

By now, you were likely in touch with the ObjectBox team and have access to Sync Server and potentially a special Sync Client version. For some platforms, we maintain packages that you can include as dependencies.

// This will try to add native dependency automatically:
apply plugin: 'io.objectbox.sync'  // instead of 'io.objectbox'

dependencies {
  // Add a ":sync@aar" suffix to the version
  implementation "io.objectbox:objectbox-android:$objectboxVersion:sync@aar"
}
​
// Apply plugin after dependencies block so they are not overwritten.
apply plugin: 'io.objectbox.sync'

dependencies:
  objectbox: ^0.9.0
  // use objectbox_sync_flutter_libs instead of objectbox_flutter_libs
  objectbox_sync_flutter_libs: any

Please reach out to the ObjectBox team

Now it's time to verify the setup using a flag telling if Sync is available; for example, simply log the result:

import io.objectbox.sync.Sync
​
var syncAvailable = if (Sync.isAvailable()) "available" else "unavailable"
Log.d(App.TAG, "ObjectBox Sync is $syncAvailable")

import io.objectbox.sync.Sync;
​
String syncAvailable = Sync.isAvailable() ? "available" : "unavailable";
System.out.println("ObjectBox Sync is " + syncAvailable);

Sync.isAvailable()

// Depending on the platform something like:
Sync.isAvailable()

Enable your Objects for ObjectBox Sync

ObjectBox Sync allows you to define which objects are synced and which are not. This is done at an object type level (a "class" in many programming languages). By default, an object (type) is local only: objects are kept in the database on the local device and do not get synced to other devices.

To enable sync for an object type, you add a "sync" annotation to the type definition. This is typically the entity source file, or, if you are using ObjectBox Generator, the FlatBuffers schema file:

@Sync @Entity
public class User {
    ...
}

// objectbox: sync
class User: Entity {
    ...
}

@Entity
@Sync
class User {
    ...
}

/// objectbox: sync
table User {
    ...
}

If you already have a non-synced type that you now want to sync (see also the info box above), these are the typical options you have:

1. If you are still in development, add the sync annotation and wipe your database(s) to start fresh with that new data model

2. "Replace" the entity type using a new UID (check schema changes docs for the ObjectBox binding you are using). You can keep the type name; to ObjectBox it will be a different type as the UID is different. This will delete all existing data in that type.

3. Have a second, synced, object type and migrate your data in your code following your rules.

Start the Sync Client

Create a Sync client for your Store and start it. It connects to a given sync server URL using some form of credentials to authenticate with the server. A minimal setup can look like this:

SyncClient syncClient = Sync.client(
        boxStore, 
        "wss://127.0.0.1" /* Use ws for unencrypted traffic. */, 
        SyncCredentials.sharedSecret("<secret>")
).buildAndStart(); // Connect and start syncing.

​

try Sync.makeClient(store: store, urlString: "ws://127.0.0.1:9999",
  credentials: SyncCredentials.makeSharedSecret(string: "<secret>"))
try client.start()

SyncClient syncClient = Sync.client(
        store,
        'wss://127.0.0.1:9999', // wss for SSL, ws for unencrypted traffic
        SyncCredentials.sharedSecretString('<secret>'));
syncClient.start(); // connect and start syncing

std::shared_ptr<obx::SyncClient> syncClient = obx::Sync::client(
    store, 
    "wss://127.0.0.1:9999",  // wss for SSL, ws for unencrypted traffic
    obx::SyncCredentials::sharedSecret(sharedSecretString)
);
syncClient->start();  // connect and start syncing

OBX_sync* sync_client = obx_sync(store, "wss://127.0.0.1:9999");  // wss for SSL
obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_SHARED_SECRET, 
    shared_secret_string, 
    strlen(shared_secret_string)
);
obx_sync_start(sync_client);  // connect and start syncing

Sync client is started by calling start()/buildAndStart(). It will then try to connect to the server, authenticate and start syncing. Read below for more configuration options you can use before starting the connection.

Once the client is logged in, the server will push any changes it has missed. The server will also push any future changes while the client remains connected. This sync updates behavior can be configured.

All of this happens asynchronously. To observe these events (log in, sync completed, …) read below on how to configure an event listener.

The client will now also push changes to the server for each Store transaction.

Should the client get disconnected, e.g. due to internet connection issues, it will automatically try to reconnect and resume syncing.

Secure Connection

When using wss as the protocol in the server URL a TLS encrypted connection is established. Use ws instead to turn off transport encryption (insecure, not recommended! e.g. only use for testing).

Authentication options

These are the currently supported options for authentication with a sync server:

Shared secret

SyncCredentials credentials = SyncCredentials.sharedSecret("<secret>");

// use a string
SyncCredentials credentials = SyncCredentials.sharedSecretString("<secret>");
​
// or a byte vector
Uint8List secret = Uint8List.fromList([0, 46, 79, 193, 185, 65, 73, 239, 15, 5]);
SyncCredentials credentials = SyncCredentials.sharedSecretUint8List(secret);

// use a string
obx::SyncCredentials creds = obx::SyncCredentials::sharedSecret("string");
​
// or a byte vector
std::vector<uint8_t> secret = {0, 46, 79, 193, 185, 65, 73, 239, 15, 5, 189, 186};
obx::SyncCredentials creds = obx::SyncCredentials::sharedSecret(std::move(secret));

// use a string
const char* secret = "secret"
obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_SHARED_SECRET, 
    secret, 
    strlen(secret)
);
​
// or a byte vector
uint8_t secret[] = {0, 46, 79, 193, 185, 65, 73, 239, 15, 5, 189, 186};
obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_SHARED_SECRET, 
    secret, 
    sizeof(secret)
);

This can be any pre-shared secret string or a byte sequence.

Google Sign-In

SyncCredentials credentials = SyncCredentials.google(account.getIdToken());

// use a string
SyncCredentials credentials = SyncCredentials.googleAuthString("<secret>");
​
// or a byte vector
Uint8List secret = Uint8List.fromList([0, 46, 79, 193, 185, 65, 73, 239, 15, 5]);
SyncCredentials credentials = SyncCredentials.googleAuthUint8List(secret);

obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_GOOGLE_AUTH, 
    googleIdToken, 
    strlen(googleIdToken)
 );

The ObjectBox sync server supports authenticating users using their Google account. This assumes Google Sign-In is integrated into the app and it has obtained the user's ID token.

No authentication (insecure)

SyncCredentials credentials = SyncCredentials.none();

SyncCredentials credentials = SyncCredentials.none();

obx::SyncCredentials credentials = obx::SyncCredentials::none();

For development and testing it is often easier to just have no authentication at all to quickly get things up and running.

Manually start

Using the example above, the sync client automatically connects to the server and starts to sync. It is also possible to just build the client and then start to sync once your code is ready to.

// Just build the client.
SyncClient syncClient = Sync.client(...).build();
​
// Start now.
syncClient.start();

Note that a started sync client can not be started again. Stop it before starting it again.

Listening to events

The sync client supports listening to various events, e.g. if authentication has failed or if the client was disconnected from the server. This enables other components of an app, like the user interface, to react accordingly.

SyncLoginListener loginListener = new SyncLoginListener() {
    @Override
    public void onLoggedIn() {
        // Login succesful.
    }
​
    @Override
    public void onLoginFailed(long syncLoginCode) {
        // Login failed. Returns one of SyncLoginCodes.
    }
};
​
SyncCompletedListener completedListener = new SyncCompletedListener() {
    @Override
    public void onUpdatesCompleted() {
        // A sync has completed, client is up-to-date.
    }
};
​
SyncConnectionListener connectListener = new SyncConnectionListener() {
    @Override
    public void onDisconnected() {
        // Client disconnected from the server.
        // Depending on the configuration it will try to re-connect.
    }
};
​
// Set listeners when building the client.
SyncClient syncClient = Sync.client(...)
  .loginListener(loginListener)
  .completedListener(completedListener)
  .connectionListener(connectListener)
  .build();
  
// Set (or replace) a listener later.
syncClient.setSyncLoginListener(listener);
​
// Remove an existing listener.
syncClient.setSyncConnectionListener(null);

// use a non-capturing lambda or a static/global function (see C example)
auto loginListener = [](void* arg) {
    (*(int*) arg)++;
};
​
// the arg is passed back a listener and can be anything, e.g. a class instance
int loginListenerArg = 0;
syncClient->setLoginListener(loginListener, &loginListenerArg);
​
// there can be only one listener of a given type, so calling again with a 
// different callback changes the listener (unassignes the previous one)
syncClient->setLoginListener(..., ...);
​
// reset (remove) a listener
syncClient->setLoginListener(nullptr, nullptr);

void login_listener(void* arg) {
    (*(int*) arg)++;
}
​
void main() {
    ...
    int login_listener_arg = 0;
    obx_sync_listener_login(sync_client, login_listener, &login_listener_arg);
}
​
// there can be only one listener of a given type, so calling again with a 
// different callback changes the listener (unassignes the previous one)
obx_sync_listener_login(sync_client, ..., ...);
​
// reset (remove) a listener
obx_sync_listener_login(sync_client, NULL, NULL);

Advanced

Listening to incoming data changes

For advanced use cases, it might be useful to know exactly which objects have changed during an incoming sync update. This is typically not necessary, as observing a box or a query may be easier.

On each sync update received on the client, the listener is called with an array of "Sync Change" objects, one for each affected entity type. It includes a list of affected object IDs - the ones that were put or removed in the incoming update.

SyncChangesListener changeListener = syncChanges -> {
    for (SyncChange syncChange : syncChanges) {
        // This is equal to Example_.__ENTITY_ID.
        long entityId = syncChange.getEntityTypeId();
        // The @Id values of changed and removed entities.
        long[] changed = syncChange.getChangedIds();
        long[] removed = syncChange.getRemovedIds();
    }
};
// Set the listener when building the client.
syncBuilder.changeListener(changeListener);
// Or set the listener later.
syncClient.setSyncChangeListener(changeListener);
​
// Calling again replaces an existing listener.
syncClient.setSyncChangeListener(changeListener);
// Remove an existing listener.
syncClient.setSyncChangeListener(null);

​
/// Sample listener collecting all puts and removals
class StatsCollector {
    struct EntityChanges {
        std::vector<obx_id> puts;
        std::vector<obx_id> removals;
    };
    std::unordered_map<obx_schema_id, EntityChanges> statsPerEntity;
​
    /// Receives changes on the object instance, forwarded by the static forward().
    void onChanges(const OBX_sync_change_array* changes) {
        for (size_t i = 0; i < changes->count; i++) {
            const OBX_sync_change& change = changes->list[i];
            EntityChanges& stats = statsPerEntity[change.entity_id];
            if (change.puts) collect(change.puts, stats.puts);
            if (change.removals) collect(change.puts, stats.removals);
        }
    }
​
    /// Update given vector by adding all ids from current change list.
    void collect(const OBX_id_array* ids, std::vector<obx_id>& targetVector) {
        targetVector.reserve(targetVector.size() + ids->count);
        for (size_t i = 0; i < ids->count; i++) {
            targetVector.push_back(ids->ids[i]);
        }
    }
​
public:
    /// Just forwards the C-callback to the instance of this class.
    static void forward(void* arg, const OBX_sync_change_array* changes) {
        static_cast<StatsCollector*>(arg)->onChanges(changes);
    }
};
​
​
void main() {
    ...
    StatsCollector collector;
    syncClient->setChangeListener(StatsCollector::forward, &collector);
}

void on_puts(void* arg, obx_schema_id entity_id, const OBX_id_array* ids) {
   //...
}
​
void on_removals(void* arg, obx_schema_id entity_id, const OBX_id_array* ids) {
    //...
}
​
void change_listener(void* arg, const OBX_sync_change_array* changes) {
    for (size_t i = 0; i < changes->count; i++) {
        const OBX_sync_change* change = &changes->list[i];
        if (change->puts) {
            on_puts(arg, change->entity_id, change->puts);
        }
        if (change->removals) {
            on_removals(arg, change->entity_id, change->removals);
        }
    }
};
​
void main() {
    ...
    obx_sync_listener_login(sync_client, change_listener, &change_listener_arg);
}

Listeners concurrency

Some events may be issued in parallel, from multiple background threads. To help you understand when and how you need to take care of concurrency (e.g. use mutex/atomic variables), we've grouped the sync listeners to these two groups:

• ​State listeners - listening to login success/failure, connection status, sync complete.

• ​Data change listener - listening to incoming data changes.

There can be only one event executed at any single moment from a listener in a single group. You can imagine this as if there were two parallel threads, one could only issue "state" events, the other only "data change" events.

Controlling sync updates behavior

By default, after the Sync client is logged in, its database is updated from the server and the client will automatically subscribe for any future changes. For advanced use cases, like unit testing, it is possible to control when the client receives data updates from the server.

To change the default behavior, configure the "Request Updates Mode" before starting the client connection. Three modes are available:

• automatic (default): receives updates on login and subscribes for future updates.

• automatic, but no pushes: receives updates on login but doesn't subscribe for future updates.

• manual: no automatic updates on login or on any updates in the future.

When using one of the non-default modes, synchronization can be controlled after login during application runtime by requesting and cancelling updates using the client:

syncClient = syncBuilder
        // Turn off automatic sync updates.
        .requestUpdatesMode(RequestUpdatesMode.MANUAL)
        .build();
​
// Wait for login attempt, proceed if logged in.
syncClient.awaitFirstLogin(20 * 1000 /* ms */);
if (syncClient.isLoggedIn()) {
    // Turn on automatic sync updates.
    syncClient.requestUpdates();
    
    // Turn off automatic sync updates, cancel ongoing sync.
    syncClient.cancelUpdates();
    
    // Request one-time update.
    // Will update client with latest data.
    syncClient.requestUpdatesOnce();
}

std::shared_ptr<obx::SyncClient> syncClient = obx::Sync::client(store, ...);
syncClient->setRequestUpdatesMode(OBXRequestUpdatesMode_MANUAL);
syncClient->start();  // Connect but don't synchronize yet.
​
// Turn on sync updates and subscribe for pushes.
syncClient->requestUpdates(true);
​
// Cancel ongoing synchronization & unsubscribe from future updates.
syncClient->cancelUpdates();
​
// Alternatively, catch up with the server but don't subscribe for future.
// You can call this instead of subscribing to do one-time updates as needed.
syncClient->requestUpdates(false);

OBX_sync* sync_client = obx_sync(store, ...);
obx_sync_credentials(sync_client, ...);
obx_sync_request_updates_mode(sync_client, OBXRequestUpdatesMode_MANUAL);
obx_sync_start(sync_client);  // Connect but don't synchronize yet.
​
// Turn on sync updates and subscribe for pushes.
obx_sync_updates_request(sync_client, true);
​
// Cancel ongoing synchronization & unsubscribe from future updates.
obx_sync_updates_cancel(sync_client);
​
// Alternatively, catch up with the server but don't subscribe for future.
// You can call this instead of subscribing to do one-time updates as needed.
obx_sync_updates_request(sync_client, false);