There were couple of events — that will be described below — that led me to port postgresql-async from Scala to jasync-sql in Kotlin last week. There are still lots of missing pieces, but an alpha release is available and work is still being done.
In this post I wanted to share how I converted the code from Scala to Kotlin and what I learned from it, so it can help other developers tackling the same task.
But first, Why? (And after that, how?)
Why?
I moved to a new team at Outbrain, and one of my tasks is to orchestrate upgrade of our modules from Scala 2.10 to 2.11. It turns out that it is possible, but it is a pain in the ass since it will require us to “patch” all our JVM modules for multiple artifacts as described here. Even our Java modules! since all of them depend on ob1k-db which depends on pstgresql-async which — in turn — depends on Scala 2.10/2.11 with different artifact.
So it might be a good idea to remove that Scala dependency that we have in all out modules... You know how much I like Scala and Kotlin from my previous post:
Scala--pack your bags; Kotlin is coming!_TL;DR — Many people ask me why do I think Kotlin is better than Scala or claims the opposite, so in this post I will…_medium.com
In addition, last week, after more than one year of inactivity, there was finally a commit approving that postgres-sql is not being maintained anymore. This was the last straw.
In addition, we are using the MySQL async flavor of the lib and didn’t find a drop-in replacement for that.
One big advantage is that Scala and Kotlin are very similar — feature and syntax wise — so it is relatively tempting to try and port the code.
How?
Before we dive into all those greasy syntax details take a break and contribute to that new open source library by a visit and a star 😉:
jasync-sql/jasync-sql_Async, Netty based, JVM database drivers for PostgreSQL and MySQL written in Kotlin - jasync-sql/jasync-sql_github.com
The conversion itself is divided into 2 main steps:
- Automatic line by line find and replace script to save some time consuming monkey typing.
- Reviewing the files manually and fixing all compilation errors, taking decisions how to convert and also improving the script.
The Script
It is really a simple and stupid kscript, maybe even embarrassingly stupid. Some of the lines even do not being replaced to a valid full statement: see pattern matching and casting for example.
I didn’t have the time nor the expertise to use something like antlr and write a parser or a full blown converter, plus I had some very tailored and specific needs. But you are more than welcome to do that.
So, without further ado, here is a simplified/cleaned version of the script:
The script is a kscript, that gets one argument: either a .kt file which is a Scala file that was already renamed, or a directory to convert files in it recursively.
The script does a simple line by line find and replace: def to fun, trait to interface etc’. Nothing fancy. But as I mentioned before the fact that the languages has similar syntax helps. Convert to Java for example can be more complex.
Lessons learned / decisions I took
The reason I wrote this blog post is remind myself what I did. Some files still need to be converted and other people are also contributing so this can also help.
The rest is just a list of items in no particular order, and might be also updated in the future.
Future -> CompletableFuture
The original code uses Scala Future extensively and I had to find an alternative. And there were a lot:
- Netty future — syntax seems verbose and outdated.
- JavaRX/Guava/ Other lib future — require another external dependency.
- Java 8 Completable Future — had to depend on Java 8 at least.
- Kotlin deferred —used mainly for couroutines, so not as rich and not sure how comfortable for java users . Was a bit hard for me to find how to compose it.
Decision was to use CompletableFuture as this is mainly a backend lib, means I don’t see a reason to use reactive relational-sql lib in Android, and Java 8 is widely used other than in Android.
Note that CompletableFuture replace Scala Future and Promise.
Dependencies
Since this is a sort of a driver library, I tried to minimize the number of external dependencies and it affected other decisions about usage.
Finalize
It turns out you don’t have to override finalize method in Kotlin.
Data Structures
I don’t remember all, but here is the conversion I did and do remember:
- Seq -> List
- IndexedSeq -> List
- ArrayBuffer -> MutableList
Bit fiddling
Kotlin is a bit weird with byte as it don’t have all operators on it yet. Some of the classes I converted to Java, and other I left in Kotlin and hope that I got it right as I am not 100% sure how Scala handled that. comments are welcome on that.
Extension Method & Properties
I didn’t get it from the beginning, but at some point I realized I can make Kotlin very similar to Scala with extensions and that was very cool.
For example Kotlin has size on List and in Scala it is length.
Problem? Extension.
Try
I decided to port/use a similar class from Scala+Arrow.
Braces in methods declaration and call
Scala do not enforce it and it is very confusing and pain to convert sometimes.
Duration -> Duration
Decided to use java.util.Duration
Execution context and implicit parameters
I found that feature very confusing, so I changed all implicit parameters to mandatory. It make the code a bit more verbose, but much more clear IMHO.
I used the common pool as default execution context although in ob1k we use another one and we just pass it explicitly anyway.
Tests
The original lib uses specs2. Originally I thought to leave them in Scala for a while, but that seems like a lot of work anyway since a lot of internal code was changed. Conversion is still WIP thanks to mostly other contributors.
Option
I mostly replaced that with nullable types, with some extension helpers: https://github.com/jasync-sql/jasync-sql/blob/master/db-async-common/src/main/java/com/github/jasync/sql/db/util/NullableUtils.kt
Here I saw Kotlin way as a better approach because in Scala sometimes Option was used, but sometimes null’s were also directly used.
It is also possible to replace it with java Optional.
Version -> KotlinVersion
There was a specific logic for it but it seems pretty standard so I found the KotlinVersion class to match that.
Implicit conversions
The root of all evil (together with premature optimization). It turns out in our case it was pretty easy to replace usage with Extension methods and Java static methods. The example can be seen here in line 25 we implicitly convert ByteBuf to ChannelWrapper by the method defined here in line 25. In Kotlin I used extension methods on ByteBuf like here, and made ChannelWrapper with static methods.
Traits -> interface + by class delegate
It turns out traits are just a replacements for multiple inheritance as they can have state. I managed to replace that with class delegation (line 55). The downside is that implementation requires methods that throws exception which can fail at runtime if were not overridden. see line 51 here.
That’s it. Thanks for reading.
As always, comments are welcome!
Title photo by Andras Vas from Unsplash