Compiler CLI API

[XAMPPRocky]

The compiler should be able to be used as a standalone executable to compile ASN.1 to a variety of languages (currently just Rust). Ideally we'd have a much more fully featured CLI for generating code. This issue requires some research and finding prior art to see if there's anything rasn should adopt.

[repnop]

https://github.com/vlm/asn1c is probably the most well-known ASN.1 compiler, generating C code from ASN.1 modules and seems to have quite a comprehensive user's guide (PDF warning) which includes its CLI options on pages 13-15, though most of them seem aimed at debugging the compiler in various ways. it also seems to generate code for all(?) encoding types by default, but I think that isn't particularly necessary since the trait system we have in Rust is suitable to take up the role of "pick your encoding", so it looks like the most common and basic usage is simply passing the ASN.1 modules to be compiled.

[Nicceboy]

I have been looking closely compilers in the past weeks, and asn1c seems to be behind the features of the current ASN.1 standard and it is not really maintained anymore.

There is a bit more active fork: https://github.com/mouse07410/asn1c

There is also European Space Agency maintained compiler for embedded systems, which is very active and might present state of the art for open-source compiling of C and Ada. Unfortunately, they don't have many encoders: https://github.com/ttsiodras/asn1scc since they don't need them, but there is an idea how to support multiple languages. It is actually written mostly with F# which is interesting.

[Nicceboy]

There is also this excellent Rust compiler https://github.com/kellerkindt/asn1rs, and there has been a lot of work done. But that project misses features this project has and also on the other way around. If we could somehow combine these projects...

[XAMPPRocky]

In relation to supporting other programming languages, I think the constraint I would place on that, is that the rasn compiler would always generate Rust code, since that is what the rasn codecs are written in, and to support a language like C/C++, or Ada, what it should do is generate idiomatic FFI bindings to the generated Rust code. That way we can keep some of the scope down in terms of needing to support languages, and makes it easier for us to use other libraries to help generate those bindings.

But that project misses features this project has and also on the other way around

What are the features do you want that you believe are missing from rasn?

[Nicceboy]

I am currently trying to decide the project where I could implement C/OER encoding, since none supports it at the moment.

This one and the above asn1rs are my choices, and I am currently strongly considering this one.
I could open separate issue from that.

Compiler support might be a bit better on the other project, and I would also like to avoid writing custom implementation for encoder/decoder on some cases, but I don't know if that is possible.

Complier with enough support would be helpful, since I might need to implement close to one hundred object notations or more. But they are not updated that regularly, so maybe it is okay. I am worried about the lot of boiler code on implementation the decoder/encoders. Or maybe I have understood this project incorrectly.

Basically I am on process to try to support this standard and its dependencies: https://forge.etsi.org/rep/ITS/asn1/pki_ts102941/-/tree/release2/

[XAMPPRocky]

I am currently trying to decide the project where I could implement C/OER encoding, since none supports it at the moment.

An OER implementation is definitely welcome, and I would be willing to mentor you through adding it to rasn. We can open a separate issue to discuss it further but just to note on COER vs OER, what rasn does in other codecs is that the decoder is always the basic variant, and the encoder is always canonical, this means that we can accept any possible value, and that there's always one representation for a encoded value.

Compiler support might be a bit better on the other project, and I would also like to avoid writing custom implementation for encoder/decoder on some cases, but I don't know if that is possible.

Yeah I don't think either compiler supports parameterisation (generics) at the moment, so you might have to write custom definitions. For what it's worth though, there are advantages to that with rasn, because with rasn you can write the definitions as a separate library, and if you shared those definitions (either in open source or with other projects) people would be able to build on-top of those definitions, and you would be able to use those improvements in your project because they have the same type, where as with the compiler based approach what ends up happening is everyone has their own auto-generated definition, and types between those two projects aren't compatible without requiring you write custom conversion code anyway.

That's why rasn is designed around the Rust type system, so that the types you make are as easy to share and re-use in Rust as they in ASN.1, even if right now it takes a some extra effort since there's no compiler.

[Nicceboy]

An OER implementation is definitely welcome, and I would be willing to mentor you through adding it to rasn. We can open a separate issue to discuss it further but just to note on COER vs OER, what rasn does in other codecs is that the decoder is always the basic variant, and the encoder is always canonical, this means that we can accept any possible value, and that there's always one representation for a encoded value.

That would be really helpful. It is also okay for my use cases that only one unique encoding value is generated.

if you shared those definitions (either in open source or with other projects) people would be able to build on-top of those definitions, and you would be able to use those improvements in your project because they have the same type, where as with the compiler based approach what ends up happening is everyone has their own auto-generated definition, and types between those two projects aren't compatible without requiring you write custom conversion code anyway.

This is a good feature and I think that this selling point should be improved on the README file. It took me a while to figure out all the benefits, and I was firstly considering only compilers. A completed data structure might also improve usability and overall quality, if everyone would just focus on that, and use that. One can always hope.

[6d7a]

I have been working on a compiler that generates rasn types from ASN1. It is still a work in progress, but it already has (limited) support for parameterization. Feedback is always welcome.

[XAMPPRocky]

@6d7a Would you be interested in instead of developing it as a external crate, moving it upstream to the compiler crate in the rasn repository?

[6d7a]

I really appreciate the proposal, but I'd prefer to have it in a separate repo, since I would like to support codegen for asn1ts as a second framework at some point.

[XAMPPRocky]

since I would like to support codegen for asn1ts as a second framework at some point.

FWIW if your goal is supporting other languages like TypeScript, I am interested in providing bindings from rasn to other languages. I'd like to have it so you can use rasn from TypeScript and Python. With those bindings it would make your compiler simpler, since you can rely on a single runtime core and easily compare the output of two different languages, and use it to test against eachother without wondering if it's a bug in rasn or asn1ts.

[Nicceboy]

Even if the usage of binding would not be possible, would it make sense to design compiler modular so, that it could be merged to rasn? There is always a limited amount of resources in OSS world and making separate compiler which generates data structures of another project, fragments the community and available resources, and instead of one polished compiler there might be two incompleted ones.

[6d7a]

FWIW if your goal is supporting other languages like TypeScript, I am interested in providing bindings from rasn to other languages. I'd like to have it so you can use rasn from TypeScript and Python.

Sounds good to me. I do appreciate rasn's elegant design, so I'd be on board regarding world domination.
I gotta say that I'm not a fan of huge monorepos. Would it make sense to refactor rasn into an org with separate repos for standards, compiler, and core framework?

[XAMPPRocky]

RE: monorepos

I can understand that can easier to approach if everything is separate, but having an organisation of single purpose repositories would add a lot of overhead to maintenance that I can't really take on right now.

The monorepo allows us to share package metadata using cargo workspaces, have a single list of issues, single list of PRs, and most importantly, a single automated release process, making it easy to iterate and release.

In the future once everything feels more stable and one thing requires individual attention, we should move it to its own repository, but at least right now, being a monorepo is the only path to keeping the maintenance minimal.

[6d7a]

Okay, so then I'll just replace the whole compiler workspace with the ones from my compiler.