Calling The Digital Tune
Marriage of music and movies
The marriage of music to movies has always fascinated me. I feel that movies and videos can seem sterile and lifeless without music. Today, we seem to listen to more music with video imagery than without it. Even in my teens, well before the video boom, I would be in my room staring at an LP cover while listening to the music not realizing that I was doing this to apply imagery to music.
With this in mind I was fascinated to read an article in The Irish Times Science Today supplement, on November 23rd, about research into this marriage by The Interaction Design Centre at the University of Limerick.
Led by director Mikael Fernstrom, the department is developing a software program currently called "Abaltat Muse". In short, this program detects events on video and matches music that is harmonically compatible. The program, so far, has been researched and assembled by a several post-graduate students with a variety of programming, musicology, videography, graphic engineering and artificial intelligence skills.
How the program was born
They first attempted to build software that was artificial intelligence model of music composers of three different music genres, atmospheric new age, baroque and r&b. Next they built software to be an artificial intelligence model of a music director attempting to compose and assign music to a video film to match its colour changes and movement. To monitor and eventually control and the merging of these two models they compiled a graphical interface to show what was analysed and created. The final addition was a set of editing tools to allow human interaction with the artificial intelligence.
The end result is now a software program that can ask a video to choose from a library of music provided by the human director which it analyzes, selects and merges with the video. Through the screen interface, the human director can select the level at which the music can be manipulated. It can be configured to demand that the stock music tracks are not altered at all or allow subtle to major changes such as changing tones, pitch and tempo and emphasis. The extreme option is allowing the program to completely re-compose music to fit the film.
They overall analysis and marriage of sound to video is achieved through a digital conversion of analog video colour and movement vibrations with analog music tone and movement vibrations. Where vibrations meet a perfect marriage of the two are presented.
Finally, add the human touch
The final digital presentation by the software should never be the final production. Like all arts, the human presentation will always be essential. In this situation there would be human based editing decisions which aesthetically return the analog to the digital production and gives it life.
Today film and sound producers along with photographers are blessed with the new digital tools. Production times and costs are now much lower than they were. To think of the painful time I used to spend on tape splicing, cut and paste graphics, light boxes etc., along with the space needed and the incredible mess these tasks used to make. The end results still requires the same human artistic and production skills. The main benefit with digital technology today is more time that can be spent on creativity rather than endless time spent on repairing silly mistakes. However, I sometimes wonder about this too when we have things like hard drive failures and software corrupted by viruses.
The benefit of the "Abaltat Muse" software is the relief from the endless time it can take to sort our music that could even be near what is suited for a video. I can see this software can provide fast video to sound versions that we can learn from and get ideas from. Like all software its essential for us to tap in and create our own individual templates otherwise our audiences will point fingers and say "Aha - that was created with Abaltat" because there is bound to be a "samey" type of signature that comes from all productions it makes just like we can often say "That looks like a Speilberg film" or "That looks like a Boorman film". It is essential that people can link our craft to our name rather than the name of the software tools we use. This can be done. Never be scared of trying to break rules when you re-configure software as you can often bend software to merge with your own personality.
When and how you may get a copy
There is no public version of the "Abaltat Muse" yet but a company, currently called Meiticheol Teoranta, in Spiddal, Co. Galway is developing a retail version simply called "Albatat", which will probably become the company name.
I would have thought the company would be facing a huge challenge from the music and movie companies as this program attracts another doorway into using copyrighted productions, especially music, but I gather this has already been dealt with. I'm going to provide a blog article on this subject within a week or two.
Product release is likely before spring 2007. I suspect this may be a very expensive professional studio version. However, I am sure they would soon follow this with a set of home use versions that's within our budgets.
I am eagerly looking forward to the release of an affordable version of "Albatat" as I will find it very useful for the production of my "A Veil Between Two Worlds" documentary dvd for release with my "A Veil Between Two Worlds" book and Claire' Roche's music CD of the same name that should all be released during 2007.
The marriage of music to movies has always fascinated me. I feel that movies and videos can seem sterile and lifeless without music. Today, we seem to listen to more music with video imagery than without it. Even in my teens, well before the video boom, I would be in my room staring at an LP cover while listening to the music not realizing that I was doing this to apply imagery to music.
With this in mind I was fascinated to read an article in The Irish Times Science Today supplement, on November 23rd, about research into this marriage by The Interaction Design Centre at the University of Limerick.
Led by director Mikael Fernstrom, the department is developing a software program currently called "Abaltat Muse". In short, this program detects events on video and matches music that is harmonically compatible. The program, so far, has been researched and assembled by a several post-graduate students with a variety of programming, musicology, videography, graphic engineering and artificial intelligence skills.
How the program was born
They first attempted to build software that was artificial intelligence model of music composers of three different music genres, atmospheric new age, baroque and r&b. Next they built software to be an artificial intelligence model of a music director attempting to compose and assign music to a video film to match its colour changes and movement. To monitor and eventually control and the merging of these two models they compiled a graphical interface to show what was analysed and created. The final addition was a set of editing tools to allow human interaction with the artificial intelligence.
The end result is now a software program that can ask a video to choose from a library of music provided by the human director which it analyzes, selects and merges with the video. Through the screen interface, the human director can select the level at which the music can be manipulated. It can be configured to demand that the stock music tracks are not altered at all or allow subtle to major changes such as changing tones, pitch and tempo and emphasis. The extreme option is allowing the program to completely re-compose music to fit the film.
They overall analysis and marriage of sound to video is achieved through a digital conversion of analog video colour and movement vibrations with analog music tone and movement vibrations. Where vibrations meet a perfect marriage of the two are presented.
Finally, add the human touch
The final digital presentation by the software should never be the final production. Like all arts, the human presentation will always be essential. In this situation there would be human based editing decisions which aesthetically return the analog to the digital production and gives it life.
Today film and sound producers along with photographers are blessed with the new digital tools. Production times and costs are now much lower than they were. To think of the painful time I used to spend on tape splicing, cut and paste graphics, light boxes etc., along with the space needed and the incredible mess these tasks used to make. The end results still requires the same human artistic and production skills. The main benefit with digital technology today is more time that can be spent on creativity rather than endless time spent on repairing silly mistakes. However, I sometimes wonder about this too when we have things like hard drive failures and software corrupted by viruses.
The benefit of the "Abaltat Muse" software is the relief from the endless time it can take to sort our music that could even be near what is suited for a video. I can see this software can provide fast video to sound versions that we can learn from and get ideas from. Like all software its essential for us to tap in and create our own individual templates otherwise our audiences will point fingers and say "Aha - that was created with Abaltat" because there is bound to be a "samey" type of signature that comes from all productions it makes just like we can often say "That looks like a Speilberg film" or "That looks like a Boorman film". It is essential that people can link our craft to our name rather than the name of the software tools we use. This can be done. Never be scared of trying to break rules when you re-configure software as you can often bend software to merge with your own personality.
When and how you may get a copy
There is no public version of the "Abaltat Muse" yet but a company, currently called Meiticheol Teoranta, in Spiddal, Co. Galway is developing a retail version simply called "Albatat", which will probably become the company name.
I would have thought the company would be facing a huge challenge from the music and movie companies as this program attracts another doorway into using copyrighted productions, especially music, but I gather this has already been dealt with. I'm going to provide a blog article on this subject within a week or two.
Product release is likely before spring 2007. I suspect this may be a very expensive professional studio version. However, I am sure they would soon follow this with a set of home use versions that's within our budgets.
I am eagerly looking forward to the release of an affordable version of "Albatat" as I will find it very useful for the production of my "A Veil Between Two Worlds" documentary dvd for release with my "A Veil Between Two Worlds" book and Claire' Roche's music CD of the same name that should all be released during 2007.
Comments
Post a Comment