Graphics for JVM
tonsky.me/blog/skija/Posted Nov 17, 2020, 10:56 AM UTC
5741
tonsky.me
ha-ha! Very funny dark mode.
11
The second is the DOM. It is a horrible collection of hacks that make simple things hard and hard things impossible.
Having built UIs in both the web and using Swing, in my opinion the opposite is true. I do not miss the days of nesting six border boxes within each other, to replicate 10 lines of CSS.
Java UIs also tend to be write once. Coming back to a UI to make changes, even simple changes, can be a huge pain. Whilst pure CSS changes can be trivial on a componetised web stack (if it's componetised).
I have thought many times “if only was I drawing this control/layout directly, I would’ve finished hours ago.”
<canvas> ???
But for most UIs you shouldn't be drawing directly. Why? A big one is accessibility. It ignores screen readers, allowing the user to resize the UI (make it 4x bigger), or recolour the UI (strong contrast or to remove backgrounds for printing). It ignores copy + paste. Depending on the stack it can also result in ignoring my OS scaling.
People say the web is really bad. Yet all of these corner cases are typically flat ignored in non-web applications. Whilst the web handles them by default.
Then you have things like scrolling. Are you going to reinvent that? I hope not. What happens if your content is too large. Are you going to work out the size by hand, and do a setSize? I get that for free on a web stack.
8
Did you read article till the end? Author tells that this is a low-level library, which is used as a building block for higher-level UI libraries. widgets and accessibility are somewhere here https://github.com/JetBrains/compose-jb
3
I don't think this Skija technology is trying to oust the DOM.
But I absolutely agree with your point about the benefits of the modern DOM, and all of the problems it solves. I too am extremely reluctant to throw it out in favor of any other UI toolkit, regardless of how fast its underlying gfx-layer renders.
This is why I have longed, almost since the inception of web browsers in the late 90's, to fix (imho) the "real problem" with browsers, which is that it's stuck with Javascript. Not just the language, but also the platform/vm.
I know it will never happen but I wish Oracle could flex some Java muscle and, maybe, I dunno, buy Mozilla, and get Firefox to start properly including a JVM into the browser, along with (most critically) a high quality DOM-manipulation API.
Then we can truly enjoy the awesomeness of all JVM languages, driving the extremely capable modern DOM.
(Note I am absolutely NOT talking about bringing back Applets! That coffin must never again be opened!!)
There have been some nice attempts to mitigate the badness of Javascript-the-language in browsers -- GWT and other "transpilers", Clojurescript, etc. Some of these options are very nice indeed, but none can overcome Javascript-the-platform. That problem is tragically hard-baked into browsers.
1
Well, Google certainly tried to replace JavaScript with Dart back in 2014. It would've happened too, if not for the huge outcry from developers who didn't want Google gaining more influence over the Web. IMO, I wish it had happened. There were a lot of smart people behind the effort and Google did it's best to make it an open standard, etc... well, at least now we have Flutter.
2
So, Java2D has all of this and there are no benchmarks or even mentions of when Java2D performance is insufficient and how much better this new library is. Just a cold statement that "Graphics2D" has poor performance. When? For BufferedImages? For VolatileImages? For on screen rendering? Printing?
Also, Java2D is part of OpenJDK so why are "we" focusing on creating a new library when we could improve the existing one?
4
I am afraid that I, for one, will not grow to like cross-platform UI with widgets that aren't reusing platform native widgets. I mean those that paint and implement their own buttons, windows and an a lot of other things like keyboard shortcuts and functionality expected by users trained by the rest of the operating system they use. I believe custom UI toolkits fundamentally are a failure, a dead-end, if not for the value they provide being how the developer doesn't have to know anything about the operating system, only the toolkit, in order to offer a GUI for their application. They come off alien not just in looks but also in behaviour, and it always stumps users, beginner and advanced alike.
If I remember correctly, AWT is at least able to delegate to operating system GUI APIs, meaning as a user you shouldn't even notice the toolkit. Swing -- not so much. But last I dealt with either was 7 years ago or so. So this is just a historical record now, probably.
I agree though that there is no reason a UI toolkit that delegates to platform-native functionality, cannot be built for Java. It is probably much more out of scope for Electron though -- if you are going to add custom libraries that expose [more] OS-native controls to JavaScript, you are deviating from "Web APIs" and Electron as a platform will gradually drift apart from what Chrome or another Web browser will let you do. Which isn't necessarily bad a direction, just saying. Web APIs is a murky ground, make no mistakes about that -- leaving some of those behind, in the name of innovation or experimentation, may open some people's eyes where it counts.
Also, neither Skija nor Skia which the former binds, are UI toolkits, well I can't call those that at least -- they are basically paint APIs. Far cry from what you'd expect a GUI toolkit would give you. Arguably, the article seems to be about "graphics in Java", but lumps toolkits and painting libraries/frameworks together.
3
I think that native apps are overrated. The one I use the most are FF, Chrome, VScode, Android Studio, Krita, Blender, KDiff and many more that behave and look almost exactly the same (it not completely the same) under different OS. From my POV, this is even better, I can switch from one OS to another without needing to learn something different. If a UI is polished and intuitive it doesn't matter if it uses native desktop. For some part it might help (like the open/save dialog) but for some it doesn't matter. As long as it run fine and the app is polished enough people don't care how it is made.
23
To each their own, right?
1
The author mentions that ui toolkit is in the works, that this library is just a first part of the puzzle
3
Interestingly, on the web, native widgets were added to Firefox due to perceived user demand. Chrome forged ahead with custom widgets, and Firefox eventually dropped native widgets.
3
I am fairly sure the crowd is fairly divided on whether a browser should have native widgets or not. Add to this that probably part of the reason Firefox did away with native widgets and went Chrome route was because styling native widgets with CSS wasn't as easy in practice as it could have been in theory, and I understand the decision. Although where there is will there is way, I'd say -- it was probably a huge convenience thing for Mozilla, too.
1
What you're asking already exists, it's called SWT, and its higher level companion, JFace.
They use the native widgets of your OS.
3
I am glad to hear that. Not sure whether SWT was around when I were looking far and wide for a Java UI library that'd give me delegation to native controls, but I'll keep SWT in mind now. Not that I myself am planning to be doing any Java applications in the future, but one does never know.
1
From a developer perspective native widgets can be a huge pain due to implementation differences. Look at layouts between Chrome and Firefox as a related example. I've dealt with tons of issues with native widgets--especially dialogs. Just a couple weeks ago I saw a native dialog with a color picker but on some OSes they add a slight background behind the dialog so all the colors picked are slightly darker. Working around that can be a huge pain.
1
If catering to platform-specific idiosyncrasies, development style and paradigms, is a huge pain then sure. But then again, developing for any particular platform -- POSIX, Windows etc -- has its advantages and disadvantages. And developing for more than one platform more so. You probably won't catch me dead developing UIs for macOS or iOS, for instance -- just not my kind of development environment, although I really like Apple's HMI guidelines (read back in the day). So if I ended up in a position where some popular software I designed were to be ported to either of those OSes, I'd have to either ask someone else to "port" the UI, or resort to using a cross-platform toolkit which does some semi-intelligent approximation.
You have a fair point in that nobody wants to do more job than they think they ought to, and bugs in different platforms, or features that have quirks, definitely add to the pain -- in so far, someone tasked with abstracting things behind one common denominator can be great. In practice however, I often find these attempts lacking. So while you mention wonky dialogs, I had to work with FontForge UI, which I think was made in Python through some toolkit that invented and painted its own controls -- as a result my scrolling was wonky, and most of the things I am used to with Windows didn't work there (like using environment variables in file dialogs).
So yeah, you and I won't agree, I think, while there are cons and pros to both approaches, obviously.
1
I think Qt does a pretty good job matching the system design and behavior, but also allowing the non-native dialogs (and maybe other widgets) at the developer's discretion.
1
Not really. There's canonical example of https://desktop.telegram.org/ which is implemented in QT (and works on Mac) and https://macos.telegram.org/ which is native. The difference is huge
1
I disagree -- I can hardly stand Qt's looks and behaviour, it pretty much for my part is the poster child of the problem I was trying to describe. It's become much better, but it's not going to replace a native Windows application look and feel for me. I suppose I am in the minority though.
That said, for many Qt applications there are usually no alternatives, so I bite my teeth and endure. Thinking of Inkscape or FontForge, for example. Not sure both/either use Qt, so please take this with a grain of salt.
1
Compose is already designed in mind with native look and feel, for example the compose widget foundation knows absolutely nothing about the material design widget implementation. Even on Android people are working with swapping out the default Compose Material implementation with their own brand implementations.
1
I am sorry, I don't understand what you're alluding to here. I have no problem with an OS component that lets people swap out one widget look and feel for another, this is more about breaking user habits with your own homemade widgets, when people have been training with certain expectations and muscle memory prior. Anyway, like I said, I don't think I fully understood what you meant to say.
1
I think the web has kinda proved that no one really cares about a "native-like" UI appearance; what is more important is that the UI is intuitive and using native-components makes that a "freebie" it doesn't mean users can't learn to navigate or utilize a different style of application.
A vast majority of average computer users don't utilize keyboard shortcuts outside of copy/paste and with the mobile world gestures have come into play but those are as few as pinch/push/swipe.
IMHO what is more important is for the basic theme to be extremely consistent; the less surprises the better.
Users want smooth performance, feedback, accessibility, and the app to just work; how that is accomplished doesn't matter.
Edit: I actually want to provide some examples to back my claim here; go take a look at Microsoft's top apps https://www.microsoft.com/en-us/store/most-popular/apps/pc the top 10 don't use a native-looking UI by any remote means.
1
I think the web has kinda proved that no one really cares about a "native-like" UI appearance
I am not sure that isn't why there is so much complaint about Web applications and the state of Web development. I am fairly confident it has something to do with it, though. It's a pain in the ass at any rate to reconcile CSS controlling the look and feel of native widgets.
A vast majority of average computer users don't utilize keyboard shortcuts outside of copy/paste and with the mobile world gestures have come into play but those are as few as pinch/push/swipe.
They don't do that arguably because there has been a culture of development and breaking changes, where people are now trained to refrain from investing too much in using their keyboard or any learned workflows, knowing too well the vendor may overhaul everything without as much as a warning. We've been on this path since the end of 90's, so the rot has set in and people are smart enough to have picked up on this style of development. Why bother learning keyboard shortcuts, if they will change with next release, or at any rate be different 6 months from now? I am a proficient computer user, and even I give up on shortcuts, because they're inconsistent between applications.
IMHO what is more important is for the basic theme to be extremely consistent; the less surprises the better.
See above. I think it's a contradiction to expect a "consistent" scheme but allow inconsistency in say, keyboard-driven interface. Where does and should the consistency stop?
Users want smooth performance, feedback, accessibility, and the app to just work; how that is accomplished doesn't matter.
Expecting developers of your average shop to deliver on all those points is unrealistic, as practice also shows these things together are only ever present in the biggest and most successful products. For the rest of us, it's usually pick one or two from the list. Which is why having an operating system to take care of those things for free for you is good -- and which is why "native" widgets are beneficial. Again, there is a contradiction in your expectation and how to get there, in my opinion.
The reason the top apps on that list don't use native looking UI and them being in the top 10 has little to do with each other. Not to mention that half of the things on that list are vendor-lock-in style of apps for which it makes no sense to provide alternatives -- you will be shut out through an API key etc, and are not going to make easy money. The reasons they're as they are is at any rate not related to whether their UI is native or not. Nobody ever complained over Microsoft Office using native UI, complaints were had over inconsistency and breaking changes, in fact. Take that as you like.
1
Very cool. I like the idea of getting into desktop development without the need to learn c++ or javascript. JavaFX is great, but limited.
2
man thats ugly yellow.. please change color scheme of the site. great article tho.
2
Now someone make a library in kotlin that's like reactjs/flutter and we have a deal
2
That is quite literally Compose, linked in the article.
1
Web techs may be hacky, but there are now decades of engineering into enabling the customization of layout and appearance, and HTML is still a pretty concise way of describing layout and UI.
Any new UI toolkit will need a way to support HTML+CSS as a definition option. To me, the real challenge to a new UI toolkit is supporting almost everything that HTML+CSS support. That's probably harder than the cross-platform/native appearance issues, the file selection problem, etc.
Keep in mind I roll my eyes at "full stack developer" and think Node.JS is a cthulhuian horror.
1
To be honest, HTML is pretty terrible as a layouting engine - while I agree that it functionally have matured to an almost usable point now. The funny thing is that decade old UI toolkits were far superior already when they first appeared.
1
[deleted]
0
Why is it awkward? I’m not following.
5
[deleted]
2
This is very exciting, but I wonder why do we have to write off AWT and JavaFX's supporting graphics engine? Why is Skia so superior to them that it's not possible to get them up-to-par? They've been around for longer (at least AWT) so why not try to improve on them and perhaps make them better than Skia? Both AWT and JavaFX have canvas, so it seems to me they could be used like Skia is if only they were as performant??
I have no background in graphics so my questions may not make any sense to someone familiar with the subject, but it seems to just make sense to try to improve things before we jump to redesign things from scratch.
Also worth noting that Skia (and Skija, by extension) is only a painting API, whereas AWT and JavaFX are what you'd call "UI toolkits", in addition to offering painting primitives (I think).
Afaik the direction is more about porting Compose from Android to desktop, than writing ui toolkits in Skia, and Compose integrates well with Skia since that's what Android uses.