Flutter Desktop Support

[OmarAboulMakarem]

Hi, i was asking if its possible to support Desktop, (windows, macos).
I want to use couchbase lite with a flutter app that is going to be distributed on desktop and mobile.

To sync with couchbase server.

Or is there another way to do it on windows?

[Rudiksz]

Yes, the main purpose of this was to add Windows and macOS support, it's fairly stable and that's where I do most of the development and testing. Android dynamic libraries are less tested, iOS should work too but I didn't get around to test it.

This is the latest recommended release
https://github.com/Rudiksz/couchbase_lite_dart/releases/tag/v.0.5.0-nullsafety.2

All CouchbaseLite functionalities present in the official SDK's are implemented, and once it's production ready you'll be able to build a flutter app for win, macos, ios, android, without using platform channels
However it's needs a lot more testing (specially performance and memory testing) until it is production ready. I only used it on some hobby projects, but I am actively working on it (when I have time).

Also it is quite bleeding edge both because of the upstream native C code and because requires Dart 2.13 or higher. On the plus side it doesn't have anything Flutter specific, so you can use whatever version you like.

[OmarAboulMakarem]

Great to hear that, am excited to try it, i will try to give it a go with my current project, which hopefully i should launch it in 3 months, do you think i can depend on this version(0.5) until u can release the stable one.

and if you are releasing the production one soon, is there a rough time line?

thanks alot for the help and waiting for more great things :)

[Rudiksz]

Unfortunately, a version that I would consider stable and mature is still some ways off, because it depends mostly on the C library which is maintained by the Couchbase team, but it isn't an officially supported library yet. Recently they made some rather major changes for the upcoming couchbase lite 3 (collections and some updates to the api), that I will eventually have to integrate too.

It also depends a lot on Dart ffi, which got some breaking updates with Dart 2.13 so I had to rewrite the memory handling again.

Regarding the 0.5 version, I don't plan to do any API changes or add new features - it's pretty much feature complete. The next version bump (probably 1.0) will be for the cbl 3, but that's very much TBD.

For 0.5.x my major concern for stability is the memory management and preventing leaks on the boundary between Dart and C. There's places where I can do automatic disposal, in other cases you would have to call dispose when you're done with an object. Balancing where I should use what, and making sure I don't keep dangling C pointers and memory around is a bit tricky.
Then there's making sure that there's no weird crashes between Dart and C, because of memory and thread issues.

Second most important thing is adding asynchronous API and/or using separate threads for more complex situations.

Another thing I'd like to do is a Flutter app for (functional) testing and benchmarking the library. I have unit tests covering pretty much all the methods, but I was thinking about a small Flutter app with some "use cases" that I can run on actual devices (desktop and phones) to test and do some benchmarks.

For example, take this rather naive test. Writing 10k small documents takes around 5seconds on my desktop. That's pretty good, but it does lock the UI at the moment, and I expect it would take longer on a smartphone.

class BenchmarkWidget extends StatefulWidget {
  const BenchmarkWidget({Key? key}) : super(key: key);

  @override
  State<BenchmarkWidget> createState() => _BenchmarkWidgetState();
}

class _BenchmarkWidgetState extends State<BenchmarkWidget> {
  var account = {};
  int duration = 0;

  void generateAccounts(AppService app) async {
    var documents = [];

    for (var i = 0; i < 10000; i++) {
      var document = Document('user_$i');
      document.properties['first_name'] = faker.person.firstName();
      document.properties['last_name'] = faker.person.firstName();
      document.properties['email'] = faker.internet.email();
      document.properties['active'] = Random().nextBool();
      document.properties['auth'] = faker.jwt.valid();
      document.properties['refr'] = faker.jwt.valid();
      documents.add(document);
    }
    
    final start = DateTime.now();
    var docCount = 0;
    app.database.db.beginBatch();
    for (final document in documents) {
      app.database.db.saveDocument(document);
      docCount++;
      if (docCount % 100 == 0) {
        app.database.db.endBatch();
        app.database.db.beginBatch();
      }
    }
    app.database.db.endBatch();
    final end = DateTime.now();

    setState(() {
      duration = end.difference(start).inMilliseconds;
    });
  }

  @override
  Widget build(BuildContext context) {
    final app = App.of(context);
    return Column(
      children: [
        ElevatedButton(
          onPressed: () => generateAccounts(app),
          child: const Text('test'),
        ),
        Text(duration.toString()),
        Text(account.toString()),
      ],
    );
  }
}

Three months is a probably a reasonable timeline to go through the code and make it leak-proof. If you decide to give it a go and have time to write some tests, or just implement some concrete use cases that would help a lot making the library stable.

Weather it's good enough in three months, will probably depend on the scale of your project, the amount of your data and how critical that data is, I guess.

[OmarAboulMakarem]

Great explanation, and amazing work, and you got a point, i will give it a try and let you know. Thanks for the feedback.

Managing memory with C is definitely a hard task

My app is not data intensive and shouldnt make any load.
Hope it can work as i really like this project.

Is there a way i can donate for this project?

[Rudiksz]

Thank you. Help with testing and maybe even fixing bugs or patching memory leaks (investing time in it) is probably the best way you can donate right now.

Depending on your experience level with C and Dart's ffi, I'm happy to explain the inner workings of the library and the gotchas I faced (and still facing) when working with Dart's ffi and the C library. Last time I saw C code was 20 years ago in school, so I'm still very much learning as I go.

[OmarAboulMakarem]

Sadly i dont have any experience with both, but am working on learning Dart's FFI and looking to hire a C developer to assist me in this matter, as i have an app that is going to be working on desktop and mobile, and a key feature is implementing offine first with sync, and Couchbase are great with this. Do you have any suggestion that can make my life better than this ;p

Thanks again for the hard work

[Rudiksz]

I have looked many times in the past year, but with Flutter there is nothing that even comes close I think.
At MongoDB they are working on bringing Realm and offline sync to Flutter but it is still in developer preview (with the usual, all API might change and do not use in production). But at least they are showing interest in Flutter, unlike Couchbase.
I use MongoDB at work, and while it's shell is pretty neat, I'm not a big fan of its query language.

Another one is objectbox.io, and they have official support for Flutter, but their sync solution is basically a black box - you have to contact them to tell you how it works.

Then there's Firebase, of course.

However both Realm and ObjectBox suffer from what I think the number one biggest pain in Dart, code generation.

My goal with this library was to have a thin Dart wrapper on top of the C library and be as unopinionated as possible. Practically you can treat the couchbase-lite Documents as an object that returns a json string of your data, that you can deserialize/serialize into your Dart object using the package you prefer.

If performance becomes important in some parts of the app, the C library has an API (which I also implemented) to manipulate the documents directly in the C memory without serialiasing/deserialising them into the Dart memory. As you can guess, this part needs the most testing though.

If you look at my earlier example, you can see that I created the couchbase documents like this:

      var document = Document('user_$i');
      document.properties['first_name'] = faker.person.firstName();
      document.properties['last_name'] = faker.person.firstName();
      document.properties['email'] = faker.internet.email();
      document.properties['active'] = Random().nextBool();
      document.properties['auth'] = faker.jwt.valid();
      document.properties['refr'] = faker.jwt.valid();

This creates a document that is backed by C memory and all the properties are set directly there.
Another way to create documents is by just passing them "json encodable" data. For example, the following does the same:

var document = Document(
    'user_$i', 
    data: {
         'first_name': faker.person.firstName(),
         'last_name': faker.person.firstName(),
         'email': faker.internet.email(),
         'active': Random().nextBool(),
         'auth': faker.jwt.valid(),
         'refr': faker.jwt.valid(),
   },
);

or even

var document = Document(
    'user_$i', 
    data: '{"foo": "bar", "active" : true}',
);

So if you already have entities that are serialized to json by some mechanism, you can just create couchbase documents using their json representation and store them in the database without any extra code generation or configuration.

[OmarAboulMakarem]

You are right, i have indeed looked in Objectbox, and it is indeed a black box, and looked into Firebase which cost a hell of alot, and looked into Realm which i didnt like, and indeed the best solution is your solution and that's why am in love with this work.

i will give it a try and if something stopped me i will try to hire someone to fix it.
specially to make it work with couchbase lite 3.

Thanks for your help and i will reach to you again soon, if you would be kind send me your contact/email on [email protected]

also what do you think of these 2 libraries and how are they different from your implementation? and why didnt you use them instead of creating your own?

https://github.com/fluttercouch/fluttercouch
https://github.com/fluttercouch/couchbase_lite

[Rudiksz]

i will give it a try and if something stopped me i will try to hire someone to fix it.
specially to make it work with couchbase lite 3.

I have already implemented most of the changes for the next version, except for replicator and blob, which seem to be not compatible with the Sync Gateway 2.8, so I can't really test them.

https://github.com/fluttercouch/fluttercouch
https://github.com/fluttercouch/couchbase_lite

I believe couchbase_lite is the successor of Fluttercouch. It is the typical Flutter plugin using platform channels to communicate to the native libraries, and as such it uses the official CouchbaseLite libraries for iOS and Android and some glue code. I did use it briefly last year when I was learning Flutter, but I haven't really looked at it since I started experimenting with dart ffi last year. The biggest downside for me is that the glue code is in Java and Swift, and like most of the packages on Flutter it did lack some features I needed, so I was writing Dart, Java and Swift code in my "one language- many platforms project".

If you are only building a mobile app (iOS+Android), I believe it is a perfectly good choice, as long as it covers your use cases. Otherwise you'll have to write glue code in both Swift and Java.

At that same time both Dart's ffi was coming into alpha phase and also couchbase seemed to pick up work on their C library so I started experimenting with it. Since then, both of them matured a bit, so I'm quite confident in the future.

Currently I have less than 300 lines of custom C code, and all else is plain Dart. As long as you have a place on the hard drive you can write to, this package should just work.

For example, couchbase is adding support for building the library for raspberry PI, so in the future it will probably be possible to have some Dart console application running on some IOT device and using this as an embedded database. Or even a Flutter app running on a raspberry IOT device.

[OmarAboulMakarem]

Great Explanation, thanks alot for your help really appreciate it.

i will start working with your library and will give you feedback, and hopefully i will be able to help with the contribution

Keep up the great work :)