Hello Nim programmers,
I've written (and continue to work on) two libraries with a public C API, the purpose of which is to facilitate making GUIs in any programming language (such as Nim).
OpenWL - cross-platform top-level windowing library, with native menus, events, clipboard/DnD.
OpenDL - cross-platform drawing library (with Quartz2D/CoreText-compatible API), built on the native APIs for each platform:
Basically, everything is native so there are no intermediate dependencies like SDL. And I've been focusing on the "big 3" platforms from day 1, so nothing should feel like an afterthought. Perhaps poorly designed from the get-go, but not an afterthought ;-)
Now I know from skimming the messages here re: GUI libraries, that some of you are working on projects with similar goals. I'm not trying to dismiss any ongoing efforts in this area, just presenting another possibility for anybody who would like to create their own GUI, but doesn't want to deal with the hassle of cross-platform internals.
I'm advertising this around to the various non-C++ languages (Nim, D, Rust, etc) because I feel there is a lot of interest in creating new GUIs purely in these languages, but until now people have had to do all the platform-specific work themselves, which is time consuming and perhaps not as enjoyable as I find it :D
So these are c++ libraries, there are no nim wrappers for them?
OpenWL is kind of like glfw and SDL?
OpenDL is most exciting as I don't know if any one has done this. Is the direct drawing utils faster then just doing everything with openGL with like nanovg or cairo?
Internally they are C++/Objective-C, but with a C API for easy consumption by any other programming language known to man. Yes, somebody would have to write the FFI definitions for Nim, but given the simplicity of my API, that's a day's work at most (or just add them incrementally as you need).
OpenWL is like GLFW, except instead of OpenGL it provides a native platform context (Direct2D, NSView, or Cairo context). And I don't know much about GLFW, but in the past I noticed that most GL apps ran a loop as fast as possible, and polled for input. Whereas OpenWL is, like the windowing systems it's based on, event-driven and only does anything when some kind of input/invalidation event comes through. So it is really not suitable for games; it's sole purpose is to provide a foundation for conventional GUIs.
Speed-wise, it's usually slower - much slower - than NanoVG, but potentially much faster than Cairo. Let me explain:
macOS uses Quartz2D natively, which (as far as I know) is still a pure CPU library. They experimented with GPU acceleration many years back but (again, as far as I know) that experiment was abandoned. However, what prompted me to make my own drawing library, was that Cairo was far too slow on the Mac - from what I could gather, this had something to do with Quartz performing color space transformation on every blit (of the Cairo buffers in memory). Perhaps that issue was eventually fixed, I'm not sure. But it was still an issue in late 2017 when I started all this.
Windows uses Direct2D natively. This is GPU accelerated, and very very fast. It blows stock Cairo out of the water (unless perhaps you're using Cairo with a GL backend - but I did not mess with that and I have no idea if it's feature-complete). Basically I wanted to focus on whatever the platform-native options were, because those are most supported, and likely to look the nicest - especially where font rendering is concerned. Quoting my post on the D forums:
Now, an alternative to OpenWL/DL might be something GLFW+NanoVG. And I certainly looked into those kinds of options before going this route. But ultimately I decided against it, because I wanted to build on top of what Apple/Microsoft/GNOME are actively working on. I figured, let the big companies focus on making what they feel are the best drawing APIs for their respective platforms, and I'll just ride on their coattails (and optimizations) ...
At this time, a purely OpenGL drawing library is still a little too DIY for me. Now maybe if the Slug text rendering library were open source, and a number of people were actively trying to integrate it with NanoVG, then I could be persuaded...
As for Linux, well, there's pretty much Cairo and that's that. Again, I haven't played with the GL backend, so I don't know what its performance would be. In theory it would just be a matter of compiling Cairo with that support, and maybe using a different context constructor, and then whatever performance gains there are, could be obtained for free. That's not a huge priority but I'm certainly open to investigating it / PRs that enable it.
I do think that fluid interfaces are important, and I am going to look into the possibility of implementing (some) Core Animation functionality on Windows and Linux, because that's how Apple enables fluid animations and transitions on the desktop, given the CPU-bound nature of Quartz2D. (You draw into layers with the CPU, and then the GPU animates those layers).
I am still intrigued by your library. I find it strange that you hint that GLFW/SDL is not suitable for applications because they are not “event driven.” But almost nothing is “single frame” any more. When you hover over a button or there is a popup you want at least a couple frame of animation to for highlighting a button or fading in a popup. Many apps like music or video apps are basically animating every frame as they run. It's also really easy to turn a polling draw every frame system 100% CPU (like a game) into not polling 0% CPU (like an old app) by just sleeping frames. Something like: if animating: draw else: sleep.
It's also strange to me that you hint that OpenWL is not suitable for games. It looks like you are doing polling in your run loop as well: https://github.com/dewf/openwl/blob/0e7174df19cf4e21a3581b9a2d79d97e49904ee7/source/win32/OpenWL.cpp#L205 It’s no different from what GLFW/SDL do. I recommend creating a “poll” interface instead of a run-for-ever interface. It is what the OS gives you! Then it become suitable for games because games can sleep or draw between the pools, and any other app with animations (which is pretty much any app now) can use it.
Another reason for poll is that apps are not just graphics and input, they also have networking and you might need to poll on the select() function. You don’t know what the app maker needs to do while drawing the screen. Yes, you can use multiple threads but threaded apps are always more complex.
Here is what other libs do: Glfw: https://github.com/treeform/quickcairo/blob/master/examples/realtime_glfw.nim#L62 SDL: https://github.com/treeform/quickcairo/blob/master/examples/realtime_sdl2.nim#L56 Glut: https://github.com/treeform/quickcairo/blob/master/examples/realtime_glut.nim#L66
You are doing it the Glut way, I don’t recommend it.
Any plans for iOS or Android? Maybe mobile version of your library? Kind of like GLFW/GLFM: https://github.com/brackeen/glfm
Maybe there's a confusion of terminology here. The line of code you highlighted in my win32 runloop is a blocking waiting-for-event call, not polling. Polling would be checking something and returning more or less immediately every time - like sampling the buttons / d-pad on a joystick, or checking to see whether there's data to be read on a network socket, etc. Whereas event loops block, and potentially forever - there's no guarantee that the win32 GetMessage() ever returns. It could literally be months of human time before that call returns. That's event-driven.
Whereas - again, I don't know if GLFW is like that these days, maybe there's an event-driven mode that's meant precisely for conventional old-school GUIs - but in general, in the old days, OpenGL programs would have a main loop that sampled inputs (joystick / keyboard / etc) and redrew over and over. And that's why the framerates would be like 300+ fps, regardless of input activity, because the developer was likely needing something to animate as smoothly as possible.
Every Window system I'm supporting gives me an event-driven runloop. Windows is a bit of an odd duck, because it doesn't have it's own never-returning .Run() method. But I suspect that's a holdover from the early days (Windows 1.0 - 3.1), when there was probably more manual processing of the MSG struct. <shrug> I didn't get into win32 until the late 90s so I've always been able to get away with leaving all processing in the WindowProc callback function. But again, that GetMessage() blocks every time, for some indeterminate amount of time. If you used that in an OpenGL program, all your animation would freeze until the user triggered an event. And then it would freeze again immediately.
Qt gets around the issue you brought up (re: networking), by incorporating network events into the event loop. That's not yet part of OpenWL, but the simple way would be to create a separate thread for all networking / a given socket, and then call the wlExecuteOnMainThread() to update the GUI as necessary. But what I really need is a "post custom event" method, that allows the user to define their own event types for the main callback. Then the networking thread could post little bundles of UI state and get right back to networking with minimal delay.
As for your main assertion, that everything these days is fluidly animated: well, yes. And it remains to be seen if anybody wants to bother with a clunky old-school GUI, the likes of which I'm trying to make easier to accomplish than ever before.
I do feel that fluid animations are important, but I also want the developer to not have to think about those in quite the same way as the rest of the logic. In theory the developer should just worry about constructing and wiring a GUI as if it's just a dumb, static piece of machinery. However there should be some "out of band" configuration possible, which allows things to be animated without disrupting the business logic too much / at all.
I am going to look into various animation libraries, notably Core Animation on Mac, to try to get some ideas. Obviously something is going to be necessary, because CPU drawing on a 5K iMac is simply not fast enough for any kind of animation. That's the problem Core Animation solves; the important (but slow) drawing is done by the CPU, then you can animate layers in various ways for smooth motion, with the GPU.
But all that is icing on a cake. First, let's give these languages SOME kind of GUI, even if it's boring and static for the most part. What they have now is mostly a hodge-podge of 1-2 platform solutions, incomplete Qt bindings, GTK reliance on all platforms, grand plans but no execution, etc - just a general mess of disappointment for anybody who expects to be able to write a conventional GUI app that just works™ no matter where you run it.
But who knows. Maybe asking each language community to implement its own GUI, even with the cross-platform foundation taken care of, is still too much (because of all the disagreement and bickering about how it should be done -- a problem only solved by corporate hierarchy and regular paychecks). In that case: oh well. At the end of the day, I've made it easier to make my own GUI in the languages I care about, and that's the only outcome I have any control over.
And no plans for mobile, probably ever. They are way too different, architecturally, from the windowing systems I'm building on top of. I'm a dinosaur, and I love oldschool desktop GUIs, and pretty much loathe all things mobile. Basically I'm 20 years too late to be doing what I'm doing. It sucks, but that's life.
Now maybe if I were trying to create exactly what you're talking about - an animation-centric, GL-based user interface library (something akin to QtQuick or JavaFX), but 1) I'm not, and 2) there are already dozens if not hundreds of cross-platform mobile frameworks in existence. Maybe one of them can be forked and made to work with Nim?
Have you looked into making some QtQuick bindings for Nim? I know some exist for languages like Haskell, so it must be do-able. That sounds a lot more up your alley than what I'm working on.
Yes, this is very doable. The beauty of OpenDL is, since I'm implementing the Apple APIs, I can just link you to their documentation:
To load custom fonts, use CTFontManagerCreateFontDescriptorsFromURL and CTFontCreateWithFontDescriptor. Then you can use the font as normal.
And here is an example of those being used.
(Keep in mind that all these functions are prefixed with dl_ in this library)
To render to an offscreen buffer, you need to create a CGBitmapContext, perform your drawing / text layout in there, create an image snapshot from the bitmap context, and then finally draw that image into your current on-screen context.
Relevant functions:
(and then whatever text layout functions you need to use - either creating a single CTLine, or using a CTFrameSetter to emit CTFrame objects (multiple lines). Both lines and frames can draw themselves to any graphic context.
I don't think I have any examples of that exactly, but I do have an example where I am stroking a text outline into an offscreen bitmap, then using that as a mask when painting a gradient on-screen: text masking example
So you manage to write CoreText API like layer on top of DirectWrite on windows and pango-cairo on linux? Why did you decide to use CoreText instead of say, inventing your own api?
I can see some of it here: https://github.com/dewf/opendl/blob/218f992bebedbdb13c5236a4564d2e94a4e706fd/source/win-direct2d/classes/CT/CTFontDescriptor.h#L60
That is pretty cool.
I am interested in this because I made pure-nim typography library: https://github.com/treeform/typography
But my text will never look quite "native" on Windows or Mac.
Do you have any examples that show how different text looks on different OSes?
I will be digging through your code to learn how you did this.
Long story short I picked it because I was already committed to using a Quartz2D (aka CoreGraphics) API for drawing, and CoreText kind of came along for the ride. I don't have enough experience designing APIs of my own, especially something of that scale, so it just made sense to use what Apple had developed and perfected over a long period. Perhaps the biggest benefit is that I can always refer to the results of rendering on the Mac, to compare and verify what the Windows/Linux behavior should be. If I came up with my own unique API, it would take a lot longer to decide what functions should even exist, what their behavior should be, all the different flags and options, etc etc. Years of work, probably.
Ideally I would even have my own CoreText-compatible text layout library for all non-Mac platforms, that relies minimally on DirectWrite or Pango for certain things (maybe font loading and glyph rendering), but does the actual layout itself. Since you have experience with that, maybe you can tell me how big a project that might end up being? Again, I'm guessing years ...
I don't have any rendering comparison examples at the moment. With one exception (the example I showed you, loading from a custom font file), most of the pages of my OpenDL demo are using different fonts on different platforms, until I figure out how I'm going to unify font lookup. IIRC CoreText wants PostScript font names, but using those on Windows is a bit tricky since there's not a 1:1 map of PS names to system fonts (from my dim recollection of working on that part). I'll figure something out, but for now, loading a specific font file from disk would be the easiest way to compare.
One issue I'm having at the moment, is that Pango is giving me too-tall results when I ask for the text bounds of a string that triggers font fallback (for exotic glyphs / emoji / whatever). And I'm getting an even bigger too-big result using one Linux distro vs. another (KDE Neon vs. Debian 9), so I presume it's dependent on which fonts are installed, or some other distro-specific configuration. It's just something I need to set aside time to investigate, ask on the Pango mailing list, etc.
I tried wrapping openwl with nimterop and the nested union in the WLEvent struct threw it off. I really need to implement nested definitions support.
Anyway, does openwl compile with mingw on Windows? Was also curious how easy it would be to use openwl to load the Scintilla editor component. It will help make my Nim based text editor cross-platform. Right now, it is Windows only.
Not sure about mingw. I presume it would need some new build/project files, unless mingw can work with .vcxproj files directly? (Currently each platform has its own platform-specific project files [VS2017/XCode/CMake]) I can look into it in the near future if you want to open an issue on OpenWL.
I don't know much about Scintilla, but I looked at it once briefly and it seemed like there were various backends? So maybe by creating an OpenWL/DL backend? Seems like a lot of work, but again, I haven't looked enough into it.
“Since you have experience with that, maybe you can tell me how big a project that might end up being?”
Here is about how much time I spent on the typography library:
Stuff still not done:
It boils down to how many language you want to support. It’s easy to get to alphabet based languages, CJK is hard because of shear number of characters as you can’t load the font into memory. At this point you got like 95% of languages covered. Then it gets into right-to-left and the shaping engine and the last 5% of languages are really hard like Arabic, Hebrew, Tibetan. Typesetting them is also hard and not a solved problem. For Thai, Lao, and Khmer you can’t wrap it properly without having AI read and understand the language. No one really supports vertical languages like Mongolian, and even though CJK can be vertical it’s never like that on digital devices. You kind of have to do it language by language basis.
Well, mingw is what's used by most Nim devs. on Windows so having support for that will be useful.
Meanwhile, here's a Nim wrapper for openwl, preprocessed with nimterop and wrapped by c2nim. It compiles fine. Needed some minor edits before and after but nothing major. However, given I don't have VS, no way to test if it actually works. I tried compiling the cpp files with mingw but ran into all sorts of issues.
As for Scintilla, or any other third-party component, it needs to be easy to bring it into openwl, especially since it's the same MFC behind the scenes, or GTK/Cocoa. If everything needs porting to the framework, will make it quite difficult to get acceptance since the standard only offers so much.
@treeform: Very interesting, thank you for the breakdown.
@shashlick: Thank you for the wrapper. Do you mind if I add this to a repository for WL/DL language bindings? (It's not created yet, yours would be the first)
I agree that mingw support is important. I've created an issue for it, will dive into that later today or tomorrow.
Scintilla support interests me primarily because it would be a great way to flesh out my CoreText implementation, adding whatever's missing to enable text editing. Right now I'm busy with another feature request (enabling support for using OpenWL with audio plugins), but after that I'll be looking for the next big thing to work on.
re: "other third-party component[s]" ... are you referring to any specifically?
So I'm pretty close with the MinGW support on OpenWL, however I have a question to ask since I don't normally use this setup.
Why is it that I have to use winpty to launch my app (which spews printf()s to the console)? I only knew to do that because of the warning that you see with the git installer in Windows. My programs aren't interactive, they just print things. Do you all have to do that when running Nim programs under MSYS2?
And for that matter, when you say most Nim users are using MingGW ... are they using MSYS2? Or just the MingGW compiler itself? I have some dependencies (like boost and icu) that I wouldn't want to have to build myself. (I normally use vcpkg for Visual Studio).
Anyway, I just need to make a small fix to make sure the shared library appears in the right place for the demo app to run, then I can move on to OpenDL which <fingers crossed> won't be too difficult to get working under MSYS.
In the meantime you could try out the mingw_friendly branch of OpenWL and let me know if you have any issues. (Do note that the API demo is built separately, and when/if you do, you currently need to move the DLL into the same dir as the c_client executable, and furthermore the latter needs to be executed from the apidemo dir.
So, from the apidemo dir:
winpty <path to build dir>/c_client.exe
I use MinGW with nim on windows all the time. I have never heard of winpty or MSYS2. To get rid of the console for a windows app from nim we just pass --app:gui not sure what you need to do in C++ to do that.
I manage to get some of OpenWL working on Mac with the help of @shashlick's modified script (here: https://gist.github.com/treeform/47865e7d06694fc224f0c492cdee41d3 )
import openwl
echo wl_GetPlatform()
proc eventLoop(window: wlWindow; event: ptr WLEvent; userData: pointer): cint {.cdecl.} =
echo "here"
var opts = WLPlatformOptions()
discard wl_Init(eventLoop, addr opts)
var props = WLWindowProperties()
props.usedFields = ord(WLWindowProp_MinWidth) or ord(WLWindowProp_MinHeight) or ord(WLWindowProp_MaxWidth) or ord(WLWindowProp_MaxHeight)
props.minWidth = 300
props.minHeight = 200
props.maxWidth = 1600
props.maxHeight = 900
let
width = 800
height = 600
var mainWindow = wl_WindowCreate(cint width, cint height, "hello there, cross-platform friend āǢʥϢ۩ใ ♥☺☼", nil, addr props)
wl_WindowShow(mainWindow) # <-- segfaults
discard wl_Runloop()
I am getting SIGSEGV when trying to show the window. Is there anything extra I need to do to the window to make it showable? Like add a menu or do any of the things in platformInit? Do you have a super minimal openWL example?
So here's a minimal Mac example, current as of right now:
https://gist.github.com/dewf/d1de81fb9bbc7f1cd82d389f94f0e7e2
Bad news, though - I just pushed a mass API rename on OpenWL this morning. The example you're showing above is a mixture of old and new naming convention, so perhaps that has something to do with the segfaults? (Sorry, it needed to be done, and sooner than later! Now it's consistent with OpenDL naming, and I'm much happier with it)
By the way, how are you linking / running this? Mac is a bit weird because (AFAIK) the GUI executables need to sit in a bundle with a .plist that describes the app a bit, otherwise it won't work quite right. So what I usually end up doing is building a dummy desktop Cocoa app in Xcode, and then using the app bundle it creates as a skeleton.
And then there's the misery of getting .dylibs properly found by executables... but since you're getting this far (crashing on wl_WindowShow) then I guess you're past all that.