Working towards integrating a digital component to my painting, I’ve been experimenting with using an animated AR overlay in relation to a painting. I’ve been making AR applications for museums for a long time, but in a team with programmers. And, as part of another funded research project, I wanted to explore and evaluate a programme called Artivive; in making the work, I’d encounter problems and have to solve them that will help me with immersive outreach work with other artists new to immersive and provide an additional element to the display of my own painting.
After 4 or 5 days of playing with the software (on and off), the following workflow evolved. a)paint/select a painting, b) concept: decide which elements might animate, c)using photoshop or FX tools in After Effects to create an animation (AE use shape layers) and ensure that you are working on a transparent layer, d)import the base painting and the video of the animation into Artivive (choose which mode – one animation and you can use the base layer, more and use Pro/3D).
Nested in this is a need for quite a bit of software literacy, so the learning curve is quite steep if you’ve not used graphics and editing software before. I had a lot of trouble rendering the video in AE to include an alpha channel – eventually finding a way of doing that fairly simply with both Adobe Media Encoder and without. Trick is to output as an MP4 but also include an alpha channel in the choice of codec. The next issue was the massive video files, so I needed a piece of compression software, electing in the end, to buy Movavi (after testing it in the trial version) because it enables you to select specific file sizes, which the freeware doesn’t do. My experiment therefore carries the Movavi watermark when seen in Artivive.
Artivive requires a less than 100mb file, so quite a journey down from a 3 GB video file! AE too presented issues as I’d not used it in a while and I’d never used it for animation; took me a while to get how to alter the speed of the lines I’d drawn. Last snag was with my use of Artivive – early uploads showed black in the painting frame rather than transparency. I thought this was due to the render missing an alpha channel, but in fact, it was that I hadn’t checked ‘This video is transparent’ (doh!) which I thought would be unnecessary if the video was rendered with the alpha channel intact. Then it worked (with an alpha channel in the render).
Artivive gives 100 views per month and only 5 projects in free mode. I’ll see how much I use it (or others use it) and if it fires up ideas, if so I might buy the sub.
Here’s the experimental video of the application. The animation is quite crude and there are no niceties with the start and finish – but my aim was to go through the whole process to see what was needed in terms of assets, skills, and time. A more 3D approach to it would help the sense of the lines going around the tree (real 3D is not possible but a cheat version is possible). This will take more time as cutouts are needed and I might try animating with either Maya or Photoshop to help give more dynamism, subtlety, and liveliness next time around. Overall interesting possibilities, with quite strong limitations.
While the brush stoke ‘shimmer’ is present when I render out as a stand-alone file, when the video layers are assembled in Artivive, the animation effects are lost. Something gets lost in the import somewhere; can’t really work this out yet. Is it a quirk of After Effects (AE) not baking them in? Or something to do with Artivive’s compression? I’m going to try animating in PS next to see if that’s a) less fiddly, and b) less finicky! I did establish that if you assemble in Pro mode in Artivive each video layer can be 100mb (you’re allowed 3 video layers ), but 3D/Pro mode didn’t really affect how my animations looked in the final instance. No sound included as yet. Lots of playing and fiddling around (time gaps through renders and uploads so far from a smooth process). Interestingly the screen version above is recognised by the Artivive software so you get double animation in that mode! Next up sound layer and animating in PS to see if that bakes in the special effects more effectively. Addition: PS is much more fiddly to animate lines than is the case with AE – you have to draw each increment! No twaining! AE wins on this score.
15/09/22 I spent another chunk of time playing with different effects in After Effects. I experimented with the 3D (it’s not really 3D, but just three differently spaced flats to project onto – Captain Pugwash style). It just ended up looking odd with my line-based animation and didn’t have the effect of creating more space in front of the image when viewed. I think it might work better with more graphic/cartoon stuff but not good for my purposes here. Instead, the approach has to be about drawing lines that look like they are coming towards you (trompe l’oeil style). As a result, I’ve thinned some of the lines, played around with colour changes and put in a line that comes out of the front and thickens as it goes. Interestingly, when you view the painting in a dark space, the lines have more contrast and glow more (the render below has the background painting image bright). I’ll bear that in mind when hanging the work!
Here’s a render of the final animation.
I note that MP4 format doesn’t include the scatter/brush effect from AE effects palette. I’m considering this a wrap on the animation now, although some sound might be added – I’m submitting it to the Newlyn Society of Artists show.
Some experiments in different light conditions revealed that with my painting which is oils on board no reflections got in the way. Glass or shiny varnish/paint might make a difference if that makes reflections that distort the key features of the image.
My next task for my next ‘sprint’ for SB3 is to create an AR animation for a doll’s house that I had worked on a while back and had intended to ‘haunt’ using AR.
Tanya Krzywinska Art Blog 