Hi all!
It is my great pleasure to announce Norm version 2.0.0 🎉
I've rewritten in from scratch, because the previous architecture turned out to have unsolvable issues. There's zero backward compatibility, so if you've never used Norm before, just start using the new version and don't bother looking and previous versions, and if you do use Norm already, I'm afraid you'll have to rewrite your code.
But this sacrifice is not in vain! Norm 2.0.0 is much more production ready. It finally solves the n+1 problem, it lets you easily connect to as many DB connections as you want, and you finally can have your models anywhere you like. The behavior of DB procs is now more predictable, there are no more magic object construction on the fly. The syntax is now more explicit.
To learn the new Norm, visit the docs.
And just to give you the taste of the new, Nim 1.2.0-driven API, here's how you filter and update rows (taken from tsqlitesugar.nim):
discard @[newToy()].dup:
dbConn.select("price > ?", 50)
apply(doublePrice)
dbConn.update
Using macros everywhere also took its toll, especially in my case where I'd have a macro generating a macro generating a template generating a proc.
At some point, this complexity became unbearable as I couldn't even understand why my code works under some conditions and doesn't under others.
Working with instances, in the other hand, is a breeze. Since it's an instance, it's guaranteed to have all its fields initialized and typed, which makes it trivial to inspect. Even for nested objects. And to insert id field into models, I don't have to inject nodes into nodes with macros, I can just use inheritance.
And then this “a-ha” moment came when I really got it that models should be ref objects. Ref objects better reflect the relations between rows.
The new approach allows the developer to utilize multiple connections in an obvious way, without ugly hacks (looking at you, withCustomDb).
dup makes it simple to avoid creating mutating and not mutating variants of the same proc. Norm 2 has only mutating procs, and if you need an immutable variable, just dup your model.
I wish I would deref or at least print the result of a dup block, and I wish I could use with with more than one argument. Something like:
with db, obj:
select("age > ", 40)
delete
4. Better safe than sorry. I've deliberately removed the migration capabilities like add column, remove, rename table, etc. Turns out, migrations are never trivial, and it's always better to just write SQL manually than to try to automate it somehow.
That's very true. However, even if there is no automatic migration, it is helpful to have a tracking of which schema is in the database and which one is expected by the code.
As you generate a DDL statement that creates the tables (and in the future - maybe also indices, and/or triggers, and/or stored procs), perhaps you can use a sha1 of that statement to "name" the created/expected model, and then -- perhaps with a helper table that names the existing schema, you can verify whether the data model matches.
Migration from one schema to a another should NOT be within the scope of the main ORM in my opinion -- however, it is useful if versions of the software can ship with "migrate-HASH1-to-HASH2" scripts, many of which will be trivial -- and the applicator of which CAN be included with the ORM.
Just my thoughts and experience, mostly from using web2py (nowadays py4web) which made a lot of controversial choices, some good and some bad -- and most of those related to migration have a lot to teach by being bad ....
Congrats on the release, very interesting to hear about the changes. I'm not sure about the removal of migrations though, that will make it very hard to create any libraries/"plugins" as there won't be a standardized way of moving between versions.
My experience is only really with Django's ORM though, so I might be missing something
it is helpful to have a tracking of which schema is in the database and which one is expected by the code.
Norman is a migration manager for Norm. It lets you apply and undo migrations, preserving order. I have to update it for Norm 2, possibly rewriting it since Norm has changed a lot.
As of now, it just stores the name of the last applied migration in a file called .last. I'd rather store that information in the DB, but that requires more work. Also, I find your idea with hashes really interesting.
Doesn't this just waste performance creating a duplicate object that's going to be discarded?
To insert a row, you must first instantiate a model to hold the data for that row. So, either way, you're creating an object, you can't avoid that.
The problem is that sometimes you don't care about that object. You need it just to do this one DB operation, like insert. And if you create those objects with var, they persist after your operation is done.
This code sample demonstrates how you can get the object without introducing throwaway variables.
Also, I don't think it has any negative effect on performance. It does allocate memory for the object, but, as I said. there's no way around that.
I really appreciate the detailed explanation, this makes a lot of sense. Love the template macro, just another example of Nim providing the power to make life easier :)
Reported an issue for the foo1/foo2 problem!
Boy oh boy, did that bug fix turn into a major version bump with an API change: https://github.com/moigagoo/norm/pull/83
Thanks for reporting the issue! Apparently, the way JOIN statements were generated was just broken.
I'm happy to announce that Norm 2.2.0 has just been released.
This is quite a fat release, fixing a couple of nasty bugs:
I'd like to thank all contributors that helped by creating issues and sending PRs.
Norm 2.2.1 has just been released.
The biggest feature in this release is manual foreign key management. You can now omit Norm's automatic foreign key handling to optimize JOIN queries. The cost is that you'll have to fetch related objects manually.
Another significant milestone is that the project has migrated to testament, which allowed us to split the test suite across multiple folders thus improving their maintainability. As a bonus, here's a cute report testament generated: https://norm.nim.town/testresults.html
Thanks to all contributors, especially Regis Caillaud aka @Clonkk. They basically implemented the entire manual fk story.
It is my pleasure to announce that norm with the coming version of 2.7.0 is now ready and eagerly awaiting nim 2.0 !
It took a bit, not the least of which included making a fork of ndb that is now called lowdb with changed testing-infrastructure-setup, to do so as the relocation of db_common was giving us some issues, but it all worked out fine in the end.
Now with a test-suite than before that checks that all is well for both 1.6.10 and devel, we're looking forward to the eventual release! Though it won't change all that much for me since I'm already using it for my own sideprojects ^^