Passing serializers dynamically

[arkivanov]

It would be really useful if we could specify serializers for non-serializable properties at runtime, similarly to how we specify serializers for generic properties. An imaginatory API could look like this.

The code from a library:

interface Args

@Serializable
class ArgsHolder(@DynamicallySerializable val args: Args)

Then the library consumer's (client) code:

@Serializable
class ArgsA(val data: String) : Args

@Serializable
class ArgsB(val data: String) : Args

val argsSerializer : KSerializer<Args> = ... // Created dynamically at runtime

val serializer = ArgsHolder.serializer(ArgsSerializer)

[pdvrieze]

Do you mean that you want the ability to specify a serializer factory instead of a serializer? In any case it would require a good amount of work, not only to implement, but especially to determine what the right semantics of such would be.

[arkivanov]

An example of I want would be as follows.

In a library I have this:

interface Args

@Serializable
class Data(val Args: Args)

Also in the same library I want to serialize and deserialize Data using an internal format, which is implementation details of the library. It could be Json or CBOR or any other.

E.g. I want to expose:

fun save(data: Data, serializer: KSerializer<Data>)

So the consumers of the library would create their own implementations of Args, create a polymorphic serializer of Args and pass it to the library.

[pdvrieze]

As to formats, that is always up to the code that calls the (de)serialization. By all means create your own format.

On Data this should work already if annotated with @Polymorphic (as it is an interface that is the default) with the condition that you must provide the serializersModule. If serialization is initiated inside your library you have to provide the serializersModule to library somehow (so the format knows it). You can do this in any way you want (e.g. ArgsRegistry.registerSubType<MyArgSubtype>())

In the alternate, the library also supports generated serializers with parameters when given a type parameter. The serializer for the type parameter is resolved on creation of the relevant serializer. That can be in a consuming library (you need an ArgsHolder<T>(data: T).

[arkivanov]

Thanks for the suggestion. I'm aware of the @Polymorphic annotation and the serializersModule think. In reality my use case is a bit more complex, I just wanted to keep the example as simple as possible. But in general, I don't want to deal with serializer modules as it would make the library's API surface more complex. Also I don't want to re-instantiate Json every time. Another reason is that that the case with serializer modules is less compile-time safe, as the user will have to not forget to properly create and supply the module. That's why I submitted this feature request.

The generic type suggestion is also not suitable in my particular case. It also makes the API surface an the usage more complex than I would like it to be. E.g. ArgsHolder may contain multiple polymorphic properties, so having <T1, T2, T3, ...> is a no go for me.

That's why I created this feature request.

[pdvrieze]

@arkivanov I'm still not 100% of what you want. Another option can be the "self-serializable" data is where you store the serializer in the object. E.g:

@Serializable(MyContainer.Companion::class)
class MyContainer<T>(private val serializer: KSerializer<T>, val data: T) {

    companion object: KSerializer<MyContainer<*>> {
        override val serialDescriptor = SerialDescriptor("MyContainer<${serializer.serialDescriptor.serialName}>", serializer.descriptor)

        override fun encode(encoder: Encoder, value: MyContainer<*>) {
            serializer.encode(value.data as T)
        }

        override fun decode(decoder: Decoder): MyContainer<*> {
            return MyContainer(serializer.decode(decoder))
        }
    }
}

This certainly works (although it needs a custom serializer). It comes into place where you have code that has dynamic return types that need (de)serialization based upon the dynamic type (without polymorphism).

[arkivanov]

I want to be able to serialize polymorphic types without using serializer modules.

Another example would be as follows. Consider a library that we don't control and can't change with the following API.

fun <T> saveData(data: T, serializer: SerializationStrategy<T>)

Now I have this code in my project consuming the library.

// Module :base
interface Args

@Serializable
class ArgsHolder(val args: Args)

// Module :foo depends on :base
@Serializable
class FooArgs(val data: String) : Args

// Module :bar depends on :base
@Serializable
class BarArgs(val data: String) : Args

Finally, somewhere in my code I want to pass ArgsHolder to the library.

fun saveArgs(args: Args, serializer: KSerializer<Args>) {
    saveData(data = ArgsHolder(args), serializer = ...) // Somehow I need to pass the ArgsHolder serializer
}

I would like to do something like this, but it's not possible currently.

// Module :base
@Serializable
class ArgsHolder(@DynamicallySerializable val args: Args)

// Somewhere in my code
fun saveArgs(args: Args, serializer: KSerializer<Args>) {
    saveData(data = ArgsHolder(args), serializer = ArgsHolder.serializer(serializer))
}

Hope this explains it better. Btw, your code snippet doesn't compile.

[pdvrieze]

Btw, your code snippet doesn't compile.
Thanks, it also doesn't work even with the correct names (deserialization is an issue as somehow the deserializer must be determined).

I've had another shot at it (now with the compiler/testing involved). But notice that using a generic parameter just makes things easier. And you can always pass an explicit serializer that is constructed however you want. If you want the serializers to be transparently provided for you, you have to follow the rules though:

    @Test
    fun testSerializeArgsHolder() {
        val data = ArgsHolder(FooArgs("Foo"))
        val ser = ArgsHolderSerializer(FooArgs.serializer())
        val actual = Json.encodeToString(ser,data)
        val expected = "{\"args\":{\"data\":\"Foo\"}}"
        assertEquals(expected, actual)
    }

    @Test
    fun testSerializeTypedArgsHolder() {
        val data = TypedArgsHolder(FooArgs("Foo"))
        val actual = Json.encodeToString(data)
        val expected = "{\"args\":{\"data\":\"Foo\"}}"
        assertEquals(expected, actual)
    }

    @Serializable
    data class TypedArgsHolder<T: Args>(val args: T)

    class ArgsHolderSerializer<T: Args>(private val elemSerializer: KSerializer<T>) : KSerializer<ArgsHolder> {
        override val descriptor: SerialDescriptor = buildClassSerialDescriptor("ArgsHolder") {
            element("args", elemSerializer.descriptor)
        }

        override fun serialize(
            encoder: Encoder,
            value: ArgsHolder,
        ) {
            encoder.encodeStructure(descriptor) {
                @Suppress("UNCHECKED_CAST")
                encodeSerializableElement(descriptor, 0, (elemSerializer as SerializationStrategy<Args>), value.args)
            }
        }

        override fun deserialize(decoder: Decoder): ArgsHolder {
            return decoder.decodeStructure(descriptor) {
                var args: Args? = null
                while(true) {
                    when(val i = decodeElementIndex(descriptor)) {
                        0 -> args = decodeSerializableElement(descriptor, 0, elemSerializer)
                        CompositeDecoder.DECODE_DONE -> break
                        else -> error("Unexpected index $i")
                    }
                    break;
                }
                requireNotNull(args) { "No args in data" }
                ArgsHolder(args)
            }
        }
    }

You could store the element serializer in the value for encoding purposes (a good technique for the returns in web frameworks like ktor), but deserialization doesn't have a value, so can't actually determine the deserializer to use from the data (unless you have an alternative way that determines the appropriate child deserializer from the context).