ObjectBox Sync enabled library

The standard ObjectBox (database) library does not include an ObjectBox Sync implementation. Depending on the programming language, it will include the Sync API, but not the implementation. For example, ObjectBox Java in its standard version allows compiling using the Sync API, it won't any Sync logic due to the missing implementation.

By now, you were likely in touch with the ObjectBox team and have access to Sync Server and potentially a special Sync Client version. For some platforms, we maintain packages that you can include as dependencies.

// This will try to add native dependency automatically:
apply plugin: 'io.objectbox.sync'  // instead of 'io.objectbox'

implementation "io.objectbox:objectbox-sync-android:$objectboxVersion"
// Or for version 2.9.0 and older:
// implementation "io.objectbox:objectbox-android:$objectboxVersion:[email protected]"

source https://github.com/objectbox/objectbox-swift-spec-staging.git
​
target 'MyCoolSyncProject' do
    use_frameworks!
​
    pod 'ObjectBox', '1.5.0-sync-rc5'
end
​

dependencies:
  objectbox: ^0.13.0
  // use objectbox_sync_flutter_libs instead of objectbox_flutter_libs
  objectbox_sync_flutter_libs: any

dependencies:
  objectbox: ^0.13.0

bash <(curl -s https://raw.githubusercontent.com/objectbox/objectbox-dart/main/install.sh) --syncbash <(curl -s https://raw.githubusercontent.com/objectbox/objectbox-c/main/download.sh)

bash <(curl -s https://raw.githubusercontent.com/objectbox/objectbox-c/main/download.sh) --sync

bash <(curl -s https://raw.githubusercontent.com/objectbox/objectbox-go/main/install.sh) --sync

Now it's time to verify the setup using a flag telling if Sync is available; for example, simply log the result:

import io.objectbox.sync.Sync;
​
String syncAvailable = Sync.isAvailable() ? "available" : "unavailable";
System.out.println("ObjectBox Sync is " + syncAvailable);

import io.objectbox.sync.Sync
​
val syncAvailable = if (Sync.isAvailable()) "available" else "unavailable"
Log.d(App.TAG, "ObjectBox Sync is $syncAvailable")

Sync.isAvailable()

Sync.isAvailable()

obx::Sync::isAvailable()

obx_has_feature(OBXFeature_Sync)

objectbox.SyncIsAvailable()

// Depending on the platform something like:
Sync.isAvailable()

Enable your Objects for ObjectBox Sync

ObjectBox Sync allows you to define which objects are synced and which are not. This is done at an object type level (a "class" in many programming languages). By default, an object (type) is local only: objects are kept in the database on the local device and do not get synced to other devices.

To enable sync for an object type, you add a "sync" annotation to the type definition. This is typically the entity source file, or, if you are using ObjectBox Generator, the FlatBuffers schema file:

@Sync
@Entity
public class User {
    // ...
}

@Sync
@Entity
data class User(
        // ...
)

// objectbox: sync
class User: Entity {
    ...
}

@Entity
@Sync
class User {
    ...
}

/// objectbox: sync
table User {
    ...
}

// `objectbox:"sync"`
type User struct {
    // ...
}

If you already have a non-synced type that you now want to sync (see also the info box above), these are the typical options you have:

1. If you are still in development, add the sync annotation and wipe your database(s) to start fresh with that new data model

2. "Replace" the entity type using a new UID (check schema changes docs for the ObjectBox binding you are using). You can keep the type name; to ObjectBox it will be a different type as the UID is different. This will delete all existing data in that type.

3. Have a second, synced, object type and migrate your data in your code following your rules.

Start the Sync Client

Create a Sync client for your Store and start it. It connects to a given sync server URL using some form of credentials to authenticate with the server. A minimal setup can look like this:

SyncClient syncClient = Sync.client(
        boxStore, 
        "wss://127.0.0.1" /* Use ws for unencrypted traffic. */, 
        SyncCredentials.none()
).buildAndStart(); // Connect and start syncing.

val syncClient = Sync.client(
        boxStore,
        "wss://127.0.0.1" /* Use ws for unencrypted traffic. */,
        SyncCredentials.none()
).buildAndStart() // Connect and start syncing.

try Sync.makeClient(store: store, urlString: "ws://127.0.0.1:9999",
  credentials: SyncCredentials.makeNone())
try client.start()

SyncClient syncClient = Sync.client(
        store,
        'wss://127.0.0.1:9999', // wss for SSL, ws for unencrypted traffic
        SyncCredentials.none());
syncClient.start(); // connect and start syncing

std::shared_ptr<obx::SyncClient> syncClient = obx::Sync::client(
    store, 
    "wss://127.0.0.1:9999",  // wss for SSL, ws for unencrypted traffic
    obx::SyncCredentials::none()
);
syncClient->start();  // connect and start syncing

OBX_sync* sync_client = obx_sync(store, "wss://127.0.0.1:9999");  // wss for SSL
obx_sync_credentials(sync_client, OBXSyncCredentialsType_NONE, NULL, 0);
obx_sync_start(sync_client);  // connect and start syncing

syncClient, err := objectbox.NewSyncClient(
		store,
		"wss://127.0.0.1", // wss for SSL, ws for unencrypted traffic
		objectbox.SyncCredentialsNone())
​
if err != nil {
		err = syncClient.Start() // Connect and start syncing.
}		

Sync client is started by calling start()/buildAndStart(). It will then try to connect to the server, authenticate and start syncing. Read below for more configuration options you can use before starting the connection.

Once the client is logged in, the server will push any changes it has missed. The server will also push any future changes while the client remains connected. This sync updates behavior can be configured.

All of this happens asynchronously. To observe these events (log in, sync completed, …) read below on how to configure an event listener.

The client will now also push changes to the server for each Store transaction.

Should the client get disconnected, e.g. due to internet connection issues, it will automatically try to reconnect using an exponential backoff. Once the connection suceeds, data synchronization resumes.

Secure Connection

When using wss as the protocol in the server URL a TLS encrypted connection is established. Use ws instead to turn off transport encryption (insecure, not recommended! e.g. only use for testing).

Authentication options

These are the currently supported options for authentication with a sync server:

Shared secret

SyncCredentials credentials = SyncCredentials.sharedSecret("<secret>");

val credentials = SyncCredentials.sharedSecret("<secret>")

// use a string
SyncCredentials credentials = SyncCredentials.sharedSecretString("<secret>");
​
// or a byte vector
Uint8List secret = Uint8List.fromList([0, 46, 79, 193, 185, 65, 73, 239, 15, 5]);
SyncCredentials credentials = SyncCredentials.sharedSecretUint8List(secret);

// use a string
obx::SyncCredentials creds = obx::SyncCredentials::sharedSecret("string");
​
// or a byte vector
std::vector<uint8_t> secret = {0, 46, 79, 193, 185, 65, 73, 239, 15, 5, 189, 186};
obx::SyncCredentials creds = obx::SyncCredentials::sharedSecret(std::move(secret));

// use a string
const char* secret = "secret"
obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_SHARED_SECRET, 
    secret, 
    strlen(secret)
);
​
// or a byte vector
uint8_t secret[] = {0, 46, 79, 193, 185, 65, 73, 239, 15, 5, 189, 186};
obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_SHARED_SECRET, 
    secret, 
    sizeof(secret)
);

// use a string
var creds = objectbox.SyncCredentialsSharedSecret([]byte("string"))
​
// or a byte vector
var secret = []byte{0, 46, 79, 193, 185, 65, 73, 239, 15, 5, 189, 186}
var creds = objectbox.SyncCredentialsSharedSecret(secret)

This can be any pre-shared secret string or a byte sequence.

Google Sign-In

SyncCredentials credentials = SyncCredentials.google(account.getIdToken());

val credentials = SyncCredentials.google(account.getIdToken())

// use a string
SyncCredentials credentials = SyncCredentials.googleAuthString("<secret>");
​
// or a byte vector
Uint8List secret = Uint8List.fromList([0, 46, 79, 193, 185, 65, 73, 239, 15, 5]);
SyncCredentials credentials = SyncCredentials.googleAuthUint8List(secret);

obx_sync_credentials(sync_client, 
    OBXSyncCredentialsType_GOOGLE_AUTH, 
    googleIdToken, 
    strlen(googleIdToken)
 );

// use a string
var creds = objectbox.SyncCredentialsGoogleAuth([]byte("string"))
​
// or a byte vector
var secret = []byte{0, 46, 79, 193, 185, 65, 73, 239, 15, 5, 189, 186}
var creds = objectbox.SyncCredentialsGoogleAuth(secret)

The ObjectBox sync server supports authenticating users using their Google account. This assumes Google Sign-In is integrated into the app and it has obtained the user's ID token.

No authentication (insecure)

SyncCredentials credentials = SyncCredentials.none();

val credentials = SyncCredentials.none()

SyncCredentials credentials = SyncCredentials.none();

obx::SyncCredentials credentials = obx::SyncCredentials::none();

obx_sync_credentials(sync_client, OBXSyncCredentialsType_NONE, NULL, 0)

var creds = objectbox.SyncCredentialsNone()

For development and testing, it is often easier to just have no authentication at all to quickly get things up and running.

Manually start

Using the example above, the sync client automatically connects to the server and starts to sync. It is also possible to just build the client and then start to sync once your code is ready to.

// Just build the client.
SyncClient syncClient = Sync.client(...).build();
​
// Start now.
syncClient.start();

// Just build the client.
val syncClient = Sync.client(...).build()
​
// Start now.
syncClient.start()

Note that a started sync client can not be started again. Stop and close an existing one and build a new one instead.

Listening to events

The sync client supports listening to various events, e.g. if authentication has failed or if the client was disconnected from the server. This enables other components of an app, like the user interface, to react accordingly.

SyncLoginListener loginListener = new SyncLoginListener() {
    @Override
    public void onLoggedIn() {
        // Login succesful.
    }
​
    @Override
    public void onLoginFailed(long syncLoginCode) {
        // Login failed. Returns one of SyncLoginCodes.
    }
};
​
SyncCompletedListener completedListener = new SyncCompletedListener() {
    @Override
    public void onUpdatesCompleted() {
        // A sync has completed, client is up-to-date.
    }
};
​
SyncConnectionListener connectListener = new SyncConnectionListener() {
    @Override
    public void onDisconnected() {
        // Client disconnected from the server.
        // Depending on the configuration it will try to re-connect.
    }
};
​
// Set listeners when building the client.
SyncClient syncClient = Sync.client(...)
  .loginListener(loginListener)
  .completedListener(completedListener)
  .connectionListener(connectListener)
  .build();
  
// Set (or replace) a listener later.
syncClient.setSyncLoginListener(listener);
​
// Remove an existing listener.
syncClient.setSyncConnectionListener(null);

val loginListener: SyncLoginListener = object : SyncLoginListener {
    override fun onLoggedIn() {
        // Login succesful.
    }
​
    override fun onLoginFailed(syncLoginCode: Long) {
        // Login failed. Returns one of SyncLoginCodes.
    }
}
​
val completedListener = SyncCompletedListener {
    // A sync has completed, client is up-to-date.
}
​
val connectListener = object : SyncConnectionListener {
    override fun onDisconnected() {
        // Client disconnected from the server.
        // Depending on the configuration it will try to re-connect.
    }
}
​
// Set listeners when building the client.
val syncClient = Sync.client(...)
  .loginListener(loginListener)
  .completedListener(completedListener)
  .connectionListener(connectListener)
  .build()
  
// Set (or replace) a listener later.
syncClient.setSyncLoginListener(listener)
​
// Remove an existing listener.
syncClient.setSyncConnectionListener(null)

/// Listens to login events.
public protocol SyncLoginListener {
    /// Called on a successful login.
    /// 
    /// At this point the connection to the sync destination was established and
    /// entered an operational state, in which data can be sent both ways.
    func loggedIn()
​
    /// Called on a login failure with a `result` code specifying the issue.
    func loginFailed(result: SyncCode)
}
​
/// Listens to sync completed events.
public protocol SyncCompletedListener {
    /// Called each time a sync was "completed", in the sense that the client 
    /// caught up with the current server state. The client is "up-to-date".
    func updatesCompleted()
}
​
/// Listens to sync connection events.
public protocol SyncConnectionListener {
    /// Called when connection is established; happens before an actual login
    func connected()
​
    /// Called when the client is disconnected from the sync server, e.g. due to a network error.
    /// 
    /// Depending on the configuration, the sync client typically tries to reconnect automatically,
    /// triggering a `SyncLoginListener` again.
    func disconnected()
}
​
/// Listens to all possible sync events. See each protocol for detailed information.
public protocol SyncListener: SyncLoginListener, SyncCompletedListener, SyncChangeListener, SyncConnectionListener {
​
}

final client = Sync.client(...);
final subscription = client.loginEvents.listen((SycnLoginEvent event) { 
  if (event == SyncLoginEvent.loggedIn) print('Logged in successfully');
});
​
client.start();
​
...
​
// don't forget unsubscribe if you don't care about the events anymore
subscription.cancel();

struct LoginListener : public obx::SyncClientLoginListener {
    void loggedIn() noexcept override;
    void loginFailed(OBXSyncCode code) noexcept override;
};
​
auto loginListener = std::make_shared<LoginListener>();
syncClient->setLoginListener(loginListener);
​
// there can be only one listener of a given type, so calling again with a 
// different callback changes the listener (un-assigns the previous one)
syncClient->setLoginListener(...);
​
// reset (remove) a listener
syncClient->setLoginListener(nullptr);

void login_listener(void* arg) {
    (*(int*) arg)++;
}
​
void main() {
    ...
    int login_listener_arg = 0;
    obx_sync_listener_login(sync_client, login_listener, &login_listener_arg);
}
​
// there can be only one listener of a given type, so calling again with a 
// different callback changes the listener (un-assigns the previous one)
obx_sync_listener_login(sync_client, ..., ...);
​
// reset (remove) a listener
obx_sync_listener_login(sync_client, NULL, NULL);

syncClient.SetLoginListener(func() { println("Logged-in") })
​
// there can be only one listener of a given type, so calling again with a 
// different callback changes the listener (un-assigns the previous one)
syncClient.SetLoginListener(func() { ... })
​
// reset (remove) a listener
syncClient.SetLoginListener(nil)

Advanced

Listening to incoming data changes

For advanced use cases, it might be useful to know exactly which objects have changed during an incoming sync update. This is typically not necessary, as observing a box or a query may be easier.

On each sync update received on the client, the listener is called with an array of "Sync Change" objects, one for each affected entity type. It includes a list of affected object IDs - the ones that were put or removed in the incoming update.

SyncChangeListener changeListener = syncChanges -> {
    for (SyncChange syncChange : syncChanges) {
        // This is equal to Example_.__ENTITY_ID.
        long entityId = syncChange.getEntityTypeId();
        // The @Id values of changed and removed entities.
        long[] changed = syncChange.getChangedIds();
        long[] removed = syncChange.getRemovedIds();
    }
};
// Set the listener when building the client.
syncBuilder.changeListener(changeListener);
// Or set the listener later.
syncClient.setSyncChangeListener(changeListener);
​
// Calling again replaces an existing listener.
syncClient.setSyncChangeListener(changeListener);
// Remove an existing listener.
syncClient.setSyncChangeListener(null);

val changeListener = SyncChangeListener { syncChanges ->
    for (syncChange in syncChanges) {
        // This is equal to Example_.__ENTITY_ID.
        val entityId = syncChange.entityTypeId
        // The @Id values of changed and removed entities.
        val changed = syncChange.changedIds
        val removed = syncChange.removedIds
    }
}
// Set the listener when building the client.
syncBuilder.changeListener(changeListener)
// Or set the listener later.
syncClient.setSyncChangeListener(changeListener)
​
// Calling again replaces an existing listener.
syncClient.setSyncChangeListener(changeListener)
// Remove an existing listener.
syncClient.setSyncChangeListener(null)

final subscription = client.changeEvents
    .listen((List<SyncChange> event) => event.forEach((change) {
          print('${change.entity}(${change.entityId}) '
              'puts=${change.puts} removals=${change.removals}');
        }));
​
...
​
// don't forget unsubscribe if you don't care about the events anymore
subscription.cancel();

/// Sample listener collecting all puts and removals
class StatsCollector {
    struct EntityChanges {
        std::vector<obx_id> puts;
        std::vector<obx_id> removals;
    };
    std::unordered_map<obx_schema_id, EntityChanges> statsPerEntity;
​
    /// Receives changes on the object instance, forwarded by the static forward().
    void onChanges(const OBX_sync_change_array* changes) {
        for (size_t i = 0; i < changes->count; i++) {
            const OBX_sync_change& change = changes->list[i];
            EntityChanges& stats = statsPerEntity[change.entity_id];
            if (change.puts) collect(change.puts, stats.puts);
            if (change.removals) collect(change.puts, stats.removals);
        }
    }
​
    /// Update given vector by adding all ids from current change list.
    void collect(const OBX_id_array* ids, std::vector<obx_id>& targetVector) {
        targetVector.reserve(targetVector.size() + ids->count);
        for (size_t i = 0; i < ids->count; i++) {
            targetVector.push_back(ids->ids[i]);
        }
    }
​
public:
    /// Just forwards the C-callback to the instance of this class.
    static void forward(void* arg, const OBX_sync_change_array* changes) {
        static_cast<StatsCollector*>(arg)->onChanges(changes);
    }
};
​
​
void main() {
    ...
    StatsCollector collector;
    syncClient->setChangeListener(StatsCollector::forward, &collector);
}

void on_puts(void* arg, obx_schema_id entity_id, const OBX_id_array* ids) {
   //...
}
​
void on_removals(void* arg, obx_schema_id entity_id, const OBX_id_array* ids) {
    //...
}
​
void change_listener(void* arg, const OBX_sync_change_array* changes) {
    for (size_t i = 0; i < changes->count; i++) {
        const OBX_sync_change* change = &changes->list[i];
        if (change->puts) {
            on_puts(arg, change->entity_id, change->puts);
        }
        if (change->removals) {
            on_removals(arg, change->entity_id, change->removals);
        }
    }
};
​
void main() {
    ...
    obx_sync_listener_login(sync_client, change_listener, &change_listener_arg);
}

syncClient.SetChangeListener(func(changes []*objectbox.SyncChange) {
		fmt.Printf("received %d changes\n", len(changes))
		for i, change := range changes {
			fmt.Printf("change %d: %v\n", i, change)
​
			// change.EntityId is a "model-entity-id", e.g. we can choose to process
			// only changes on Entity `User`, with the generated `UserBinding`:
			if change.EntityId == model.UserBinding.Id {
			  fmt.Printf("put user IDs %v\n", change.Puts)
			  fmt.Printf("deleted user IDs %v\n", change.Removals)
			}
		}
	})