MultiTime? - exploring multi-directional movement in videos for VJing
http://studio.kyperjokki.fi/engine/MultiVideo
Toni Alatalo, Heikki Salo, Tomi Hurskainen et. al of Kyperjokki
MultiVideo (a working name) is an experimental system for multi-directional movement in video. The basic idea is simple: as scratching / scrubbing is a powerful basic technique in VJing, but is traditionally limited to movement along a single time-dimension (backwards / forwards), which was possible already with film and analog video, I wanted to try if it would be possible in many directions. That can be thought of as having several time-dimensions, hence a possible name for the project is also MultiTime? and it features timetracks. Alternatively, the content can be thought of simply as alternative versions of a video, or in other cases as several interconnected video clips.
The first implementation features a multivideo simply as a 2d matrix, i.e. a grid, so that instead of a film strip there is an euclidean plane. So instead of the typical next/previous frame (right/left), which are there in normal video, there are also the alternative next/previous frames in another direction (up/down). The first version of the data structure used for such multivideos is also the simplest possible: the alternative/parallel strips that form the matrix are just subdirectories in the parent directory which is the multivideo, each including a video as an image sequence. There is a tool to render such multivideos directly from Blender, and a barebones player written using Pygame, both available via the project wiki page, which also links to different kinds of example contents.
Clearly using real-time rendering, e.g. OpenGL via game engines, can give much more flexibility than pre-rendered content used here. In fact, we have been using OpenGL/game engines for VJing and have released related tools before. But there are still uses for this kind of flexibility for pre-rendered / video material too: 1. certain effects, such as picture mosaics, still need to be pre-rendered, and it is good to have more degrees of freedom in the playback and 2. this system enables using real (i.e. photographed) video shootage, albeit it must be shot and edited in a certain way to work as a multivideo. Actually, the Blender animation & rendering tool was made to test possible scenes that could later be videoed: for example there is a sunset scene, which is planned to be shot for real, and ideas about how e.g. dance movement multivideos could be shot. Easiest photovideoed multivideos would be camera movements, but that's boring.. (and in a way done in e.g. bullet time things).
We propose adding support for multivideo to VJ authoring and performance tools. A candidate tool for us is LiVES, as we've succesfully used it for editing video for performances and using it as the playback tool too. The need is also twofold: for one, we have no solution currently for editing videoed shootage for multivideo use. Also the pygame test player is not optimized at all, and can not deal with large amounts of data.MultiVideo (a working name) is an experimental system for multi-directional movement in video. The basic idea is simple: as scratching / scrubbing is a powerful basic technique in VJing, but is traditionally limited to movement along a single time-dimension (backwards / forwards), which was possible already with film and analog video, I wanted to try if it would be possible in many directions. That can be thought of as having several time-dimensions, hence a possible name for the project is also MultiTime? and it features timetracks. Alternatively, the content can be thought of simply as alternative versions of a video, or in other cases as several interconnected video clips.
The first implementation features a multivideo simply as a 2d matrix, i.e. a grid, so that instead of a film strip there is an euclidean plane. So in addition to the typical next/previous frame (right/left), which are there in normal video, there are also the alternative next/previous frames in another direction (up/down). The first version of the data structure used for such multivideos is also the simplest possible: the alternative/parallel strips that form the matrix are just subdirectories in the parent directory which is the multivideo, each including a video as an image sequence. There is a tool to render such multivideos directly from Blender, and a barebones player written using Pygame, both available via the project wiki page, which also links to different kinds of example contents. More complex data structures have been drafted, but are not really implemented yet. One target there is to support different kinds of content, e.g. such where not all frames exist to define an euclidean plane. Another is to have a system which is not that much of variations of a single video, but about interconnecting several videos, which is a more known topic in hypermedia research and already supported by e.g. the SMIL standard.
Clearly using real-time rendering, e.g. OpenGL via game engines, can give much more flexibility than pre-rendered content used here. In fact, we have been using OpenGL/game engines for VJing and have released related tools before. But there are still uses for this kind of flexibility for pre-rendered / video material too: 1. certain effects, such as picture mosaics, still need to be pre-rendered, and it is good to have more degrees of freedom in the playback and 2. this system enables using real (i.e. photographed) video shootage, albeit it must be shot and edited in a certain way to work as a multivideo. Actually, the Blender animation & rendering tool was made to test possible scenes that could later be videoed: for example there is a sunset scene, which is planned to be shot for real, and ideas about how e.g. dance movement multivideos could be shot. Easiest photovideoed multivideos would be camera movements, but that's boring.. (and in a way done in e.g. bullet time things).
We propose adding support for multivideo to VJ authoring and performance tools. A candidate tool for us is LiVES, as we've succesfully used it for editing video for performances and using it as the playback tool too. The need is also twofold: for one, we have no solution currently for editing videoed shootage for multivideo use. Also the pygame test player is not optimized at all, and can not deal with large amounts of data.
--
Kyperjokki MultiVideo, Blender, Pygame, Metapixel and possibly LiVES is support is added..