We live in age DOMINATED by audiovisual content.
Yet no serious programming language seems to come with good built in Audio/Video file Read/Process/Write support.
I was thinking that Nim could be an exception to this.
Imagine Nim being able to read/write common audio/video files with 1 line of code without having to call 3rd party libraries or .dlls.
There are tens of thousands of real-world applications that require some form of custom realtime or offline image/audio/video processing.
If Nim runs as fast as native C and had audio/video read/write capabilities baked into the core language, you could be looking at the new DARLING of the image/audio/video processing world.
This is my suggestion to any Nim developers reading this forum - please add audiovisual capabilities into the core language itself.
Everyone faced with having to code custom image/audio/video processing algos would thank you for this functionality.
Currently, many people run to less-than-ideal solutions like Matlab, Videolab, Processing to do audiovisual processing with code.
A fast compiled language like Nim with built-in read/write support for multimedia files could attract a tremendous number of people to Nim.
Thanks For Reading,
VideoBuddha
Not quite what you are proposing, but there is an opencv nimble package (only a partial wrapper, IIRC).
I wonder how realistic implementing this would be given the Copyright issues with various codecs.
I don't agree. This is an area that is great for you, and would be very interesting for a lot of other people. But not generally. Just think about video codec developers. How would a video codec developer feel in a programming language that has already decided on the builtin video codec. I think it has nothing to do in the core language, but in a library that show how powerful the core language feature is, yes of course. A programming language is not defined by the amount of featuers, but more by the amount of features it does not need because of more general core featuers that can be used to implement such a thing.
I am developing from time to time this project: https://github.com/krux02/opengl-sandbox
This has neither support for video loading, nor does it have support for audio. But I can imagine that at some point it will have exacty the feature you are proposing. But as I said, it should not be part of the standard library.
There is NO programming language right now that does both.
The programming language is either SLOW but supports audio/video nicely, or it is FAST, but getting audio/video in and out of the language is a horror to set up.
We are in 2017. The world we live in is highly audiovisual content oriented.
If you want Nim to become popular in audiovisual computing - Computer Vision for self-driving cars or Video Processing for Film & Television for example - I strongly suggest that the language comes with basic image/audio/video read/write capabilities by default.
When people ask on the internet "what programming language is good for video processing?" people would then answer "Nim is really good for that".
Please consider basic audio/video support seriously.
Nim would become the first language that is both FAST, and that can do basic audio/video processing from the moment you install it.
You would reach thousands of users quickly just due to this built-in audiovisual functionality.
Thanks!
As @krux02 mentioned, core language should be compact and flexible without diminishing its usability. Nim syntax already flexible enough so you shouldn't have any problem implementing the dsl for multimedia applications. While developing language to fill the niche is nice, it would be more nice if you can do anything with it.
Also, since the video/audio processing is your expertise, it would be nice if you write some library for it.
@videobuddha - what do you currently use (language/libraries/codecs/...)?
There are a few Nimble packages that wrap external libraries already:
A generic video/audio module that abstracted lower level libraries would be of benefit (and is what I understand as being sort of what you are wanting).
If there was a programming language called XYZ that supported direct manipulation of video/audio files, then tomorrow, when I write my own unique fantasic (in my option) new shiny video/audio file format, then the XYZ language is obsolete and useless because it doesn't support by new shiny format (just trying to highlight that of course it is not the problem of XYZ that it doesn't support my new format).
So I agree with @Krux02
but (Nim) allows (you) to implement all your dreams that you want to have
It just needs support for more file formats, and a library to abstract the access to those libraries.
@ jlp765
We video and image processing guys tend to be highly specialized in "pixel operations" and "image processing operations".
i.e. once we can get at the actual video RGB pixels or audio sample data we know what to do with them
But we suck at things like setting up video file I/O - getting a programming language to actually load and save video/audio data reliably
I have recently tried:
Many people who ask online "which language for video processing?" are typically told to look at Processing (slow) and Matlab (also slow)
Everybody is looking for the same thing - a programming language that is FAST and has easy-to-use audio/video data read/write functions.
If Nim could do this out-of-the-box, Nim would be 3 steps ahead of other programming languages when people look around for languages that can do audio/video.
This is just a suggestion of course - but it could be important if you want Nim to become a popular language for audiovisual computing tasks.
If you want Nim to become popular in audiovisual computing - Computer Vision for self-driving cars or Video Processing for Film & Television for example - I strongly suggest that the language comes with basic image/audio/video read/write capabilities by default.
Yes, we do. So why don't you go ahead and write those capabilities? We have our hands full with other things... A package manager, forum, compiler, and standard library don't write themselves!
We have lots of people pitching ideas they can't seem to implement themselves. Common problem. Other people could implement those ideas, but time and energy are finite. Also a common problem.
What we need is some sort of NimLancer broker system that would make it easy for people to hire a freelancer to implement new open source Nim libraries, compiler / stdlib pull requests, etc. The programmer only gets paid when the requirements are met, with reputable people available to judge any disputes. This would be good for the Nim community / ecosystem as a whole.
So: what are the specific requirements of the high-level video library you wanna see, and how much Bitcoin would you be willing to chip in? ;)
I for one would be willing to code very cheaply if it's with Nim and copyfree, but this particular project isn't for me. I'm very rusty, but if I could find some relatively easy paid Nim projects, I'd be able to devote more time and eventually take on more complex projects - and of course my own ideas...
I did a lot of video processing in the past.
What you want is a frameserver like Avisynth (Windows-only, multithreading an afterthought, custom scripting language) or Vapoursynth (Windows, Mac, Linux, scripting through Python)
Example Vapoursynth to flip a video:
from vapoursynth import core
video = core.ffms2.Source(source='Rule6.mkv')
video = core.std.Transpose(video)
video.set_output()
This can be forwarded to mencoder, ffmpeg, x264 or x265 encoder. There are facilities to address particular frame, use GPU for lots of thing (example page of all the denoising filters including GPU accelerated)
This is a huge undertaking between SIMD, platform specific video API, multiprocessing, the avisynth/vapoursynth API, creating fast filters (which usually needs assembler ...), mathematical/signal processing knowledge.
This is an odd request :)
Yes, there are languages that has support for audio read/write - but the point is that those languages has that support by means of packages/libraries - that kind of functionality is never in the core language.
So, if Nim doesn't have it, I guess it's because Nim users didn't need that functionality enough!
So, unless someone has that itch, and scratches it ... then Nim will never get support for video read/write. ;)
The most complete library for audio/video file reading, writing, decoding, and everything else is ffmpeg's (the library is called "libav", but there's a fork of ffmpeg called "libav" so it's really confusing; so I'll just call it "ffmpeg")
There is no other offering, free or not, that has comparable breadth of support. I have planned to c2nim it into working condition, but have not yet found the time, and even if I did, it would likely be less useful than one can expect.
There are a few issues to consider about this library -
I don't think it's a coincidence that almost no language has up-to-date ffmpeg wrappers (and those that do usually wrap the .exe rather than link to the lib); it's a moving target.
Doesn't !ffmpeg already support most of this?
Granted that ffmpeg is implemented in C(with some ASM), but it wouldn't be all that difficult to wrap. Someone crazy enough and who has enough time on their hands may be even to fully port it to Nim.
I agree with most everyone else that Video/Audio manipulation should not be a core feature. I do think it is reasonable that the core language makes it simple for someone to write libraries for such a task. I believe this is already the case judging by the fact that multiple audio/visual libraries already exist for Nim.
The codec issue that was brought up by @krux02 is a very real one. Many video compression formats are proprietary. Even the non-proprietary ones are numerous. Trying to implement core language support for this kind of thing would involve a lot of work including testing and constant feature requests. This would detract from more important concerns like bug fixes, compiler optimizations, and documentation improvements.
Nim is already taking a while to get to 1.0. Trying to tack on additional stdlib features is not what the Nim community needs to focus on at the moment.
You probably can c2nim ffms2. This is what avisynth/vapoursynth uses to store a video in a variable like in my script
from vapoursynth import core
video = core.ffms2.Source(source='Rule6.mkv')
And then it provides facility to get specific frames
const FFMS_Frame *FFMS_GetFrame(FFMS_VideoSource *V, int n, FFMS_ErrorInfo *ErrorInfo);
const FFMS_Frame *FFMS_GetFrameByTime(FFMS_VideoSource *V, double Time, FFMS_ErrorInfo *ErrorInfo);