Source
r/programming•
GhostNULL

Jai Compiler Demo: Macros and Iteration

Posted Aug 18, 2019, 8:16 AM UTC

4159
oldatbrain•

TL/DW:

0:00:00:

  • Goal: More like Lisp than C macros.
  • Motivation 1 for macros: Replace / allow implementing iterators more succintly
  • for loops have special features: < for reversing, * for dereferencing
  • Motivation 2 for macros: Replace destructors / RAII / integrate with defer

~0:12:00:

  • Macros are just functions with #expand. So all features of function are available to macros (variable parameter count, overloading, baking, ...)
  • Macros are hygienic, no name collisions
  • Macros can use ` (backtick) to access names in the expanded / caller / target scope

~0:33:00:

  • New datatype Code for syntax trees.
  • #code ... literal
  • #insert inserts Code into the body of a macro, using the expanded / caller / target scope.
  • #insert_internal inserts Code into the body of a macro, using the macro scope.

~0:45:00:

  • Back to for loops using macros: for_expansion
  • Example: Bucket-array iteration
  • break and remove support
  • Iteration variable renaming

~1:00:00:

  • More examples (Hashtable iteration, Binary tree iteration, Bit array iteration)

~1:15:00:

  • TBD
34
masklinn•

for loops have special features: < for reversing, * for dereferencing

Motivation 2 for macros: Replace destructors / RAII / integrate with defer

Sounds like CL's loop and unwind-protect respectively.

I'm queasy about defer or context managers in non-GC'd languages (which Jai supposedly is, in my understanding): in a GC'd language memory is assumed to be plentiful (relatively speaking) and the runtime wants to manage its allocation and deallocation for reasons of throughput and latency and everything, and object destruction (if any) is hooked into the "memory resource" handling, so it makes sense that you need a second separate resource-management system for other types of resources which tend to be less plentiful and / or less automatable (e.g. sockets, fds, locks, …).

For languages without GCs however it seems like a mistake on multiple levels:

  • it creates a multi-tiered resources system unnecessarily
  • it makes non-memory resources less uniform as you won't be manipulating them the same way depending whether you're using them internally or ultimately handing them off to a third party (to say nothing of conditional paths)
  • this also opens the door to more errors as using the wrong handling opens you up to memory leak or use-after-close
  • it creates a disconnect between the memory and non-memory resources used by the same object

RAII fixes all that, it provides a uniform resource-management system across memory and non-memory resources, and does the right thing whether you're the creator, and intermediate user or the final user of a given resource. The main sticking point is fallible destruction but IME defer / context managers don't usually solve that either beyond raising some sort of fault themselves, and I'd think opt-in linear (rather than affine) RAII would be a much better fix from a non-distributed reliability POV.

7
lvl5hm•

here's an old video from Jon that talks about RAII: https://www.youtube.com/watch?v=TH9VCN6UkyQ&t=1740
I hope I'm representing his thoughts well:

  • differentiation between memory and other 'resources' is good, because trying to cram reasoning about them into the same general facility makes this facility not as good as a more specific solution. he spends almost no time handling files/etc. compared to memory
  • RAII in C++ exists because of exceptions, he strongly dislikes exceptions (he explains his reasoning in the video)
  • it makes reasoning about your code harder by giving you more concerns to juggle
  • in his experience, it wastes more time than would have taken him to just debug the bugs that it prevents
  • this is mentioned by him elsewhere: RAII encourages you to think about your objects as standalone (constructing them on the heap one by one as you need them, then using destructors to free the resulting forest of things), while the correct way to structure your data for performance is to pack them into tight arrays and allocate/free in bulk
25
oldatbrain•

I don't quite follow why non-GC and defer don't mix well.

opt-in linear (rather than affine) RAII would be a much better fix

What are linear / affine RAII? Related to affine & linear types? "An affine resource can be used at most once, while a linear one must be used exactly once."

from a non-distributed reliability POV.

What does that mean?

8
masklinn•

I don't quite follow why non-GC and defer don't mix well.

It's not that they don't mix well, it's that defer seems suboptimal in that context.

What are linear / affine RAII? Related to affine & linear types? "An affine resource can be used at most once, while a linear one must be used exactly once."

Yes. Most RAII is affine, in the sense that once you stop using an RAII resource it gets automatically collected for you at the end of the scope (more or less). The issue with that things get wonky if the dtor/collector needs to report an error (exception / panic is the only option).

A "linear" RAII would require explicitly collecting resources which would allow for error reporting (so you'd have an affine system for non-faillible collection and a linear one for faillible), and the compiler requires explicitly cleaning up linear resources.

Note that "linear" RAII doesn't solve the case of needing to report errors while unwinding (with both C++ and Rust have issues with, IIRC in both it'll lead to the process just aborting entirely) but it should make the issue less likely as dtors would have little to no reason to throw / panic anymore.

from a non-distributed reliability POV.

What does that mean?

Erlang is conceptually centered around the idea that distribution is required to achieve reliability (because if you have a single thread / process / machine and it dies you've got nothing left, so you need at the very least an other one to act as supervisor). In that view, faults are rarely or never handled synchronously up the stack, instead the faulting process dies and the fault is asynchronously reported to its supervisor (possibly indirect) which decides what to do.

"Linear" RAII doesn't do anything for this conception of reliability.

5
oldatbrain•

Thanks for the explanations.

2
oldatbrain•

I'm not too familiar with CL. Is the loop macro extensible? Google didn't turn up anything about that. In what way is it similar?

2
FatalElectron•

Not so much extensible, as it is that it'll do everything you could ever want ever, and many things you never thought you'd want, from a loop construct:

http://www.lispworks.com/documentation/lw51/CLHS/Body/m_loop.htm

Personally, I'm of the belief that it's an overengineered mess that is essentially as complex as the rest of the common lisp spec.

2
oldatbrain•

OK, thanks. So CL loopis similar to Jai for because "< for reversing, * for dereferencing" also sounds like it has (too) many built-in special features.

But they are not similar because CL loop is not extensible? I could not write a new data structure and then extend loop to iterate over the elements of that data structure?

1
sapper123•

Code is my new favorite data type in Jai.

19
PanKurzcak•

There's a hack in c where you can manually fill a function pointer with machine code at runtime and execute it (provided you jump through a bunch of syscalls to allow it). I always wanted some language to implement that in an official way.

2
xampf2•

Thats what like every dynamic recompiler/vm does.

10
PanKurzcak•

Yeah, but its not an intended feature of the language is my point, nobody explicitly created syntax for you to do that.

5
hechang27•

There is function eval and module ast in python.

3
long_march•

Well these don't work at runtime either, they're macros.

3
PanKurzcak•

But the code you pass into those macros can be decided at runtime.

-1
long_march•

How does that work when the macros are expanded at compile time?

6
meneldal2•

(provided you jump through a bunch of syscalls to allow it)

Only on a modern platform, if you run your cpu without the training wheels (also known as real mode), you can do everything without any annoying syscalls.

1
oldatbrain•

Sounds similar to a C# expression tree. But C# expression trees do not support statements. And they have explicit parameters. And the types of these parameters (and the "return" value of the expression) is explicitly part of the expression's type (e.g. Expression<Func<int, bool>>).

2
brunhilda1•

There are some great ideas here that are being explored.

5
s0lly•

Very excited to see where this goes!

4
Inane_Bob•

an hour and a half long video about a closed source programming language

cool

-19
kjk•

Unreleased programming language.

It's unreleased because it's unfinished.

When it's released, it'll be open sourced.

13
miki151•

Will the source code be released in a youtube video?

21
become_relevant•

No it will be posted on pastebin :)

13
FG_Regulus•

I've seen this sub rip apart other people who post WIP projects, but because it's Blow it seems to be okay with these. He's been publishing these for several years and there's still nothing to actually play with. He doesn't even incorporate feedback from what I can tell. I don't really see the point, especially when he's mostly just making no-GC Lisp... But with some syntax sugar.

At one point I considered Jai to be really cool, but for me it's worn it's welcome and until I see a release I generally just consider it vaporware.

Of course, feel free to disagree! In a few more years I may be proven to be a big idiot and we'll all be using it!

15
shevy-ruby•

Sorry but he is right.

I don't think any of us should debate this.

Either open source a language - OR DON'T RELEASE IT AT ALL.

I think Jonathan is cool, but closed source shit is simply unacceptable. We don't live in 1980 anymore.

Even if the source code sucks, so what? I don't think Jai is a good language from a design (and C++ is designed by cthulhu-worshipping clowns) - but it simply should be a STANDARD to publish, at all times, open source, without delay and without other excuses.

So many other languages manage to do so.

I guess jai is only intended as a self-promo self-patting on the shoulder for ego boosting if it remains closed source.

-30
[deleted]•

[deleted]

23
Plasmubik•

Honestly I think shevy-ruby is just a troll. They show up in almost every language related thread to shit on any language that's not Ruby.

4
[deleted]•

[deleted]

-2
DoctorGester•

The comment you are replying to literally says it’s going to be open sourced, what are you even talking about?

13
kankyo•

Going to is a promise though. If the guy gets run over by a bus he can't fulfill the promise. It's closed source until it's open. It's not open yet so it's by definition closed. There is no quantum super position of open/closed.

-5†