4D Capture | 10 Camera Array

4dprt3-02

Time seems to flyby at the moment, I’ve been swamped on some fantastic projects over the last few weeks with more booked this year and next, so I managed to squeeze in some time to run some 4D tests which I have been meaning to do since the summer.

Using 10 DSLR’s recording Synchronized footage, the idea is to process exported frames in Agisoft with the help from Alexey Pasumansky. Alexey is developing a script to batch process the data, to then stream back in Softimage with colour information to track with.

The only draw back with using DSLR’s is the awful built in compression! it really affects 3D Stereo Extraction but I have some other Cameras on order to overcome this issue. For now the DSLR’s are pretty good. It means anyone can build a 4D Capture array for under £5k, unlike most commercial systems that can vary from £50k to £150k.

YouTube Preview Image
4D Mesh Data Processed in Agisoft Photoscan
YouTube Preview Image

Alexey kindly  processed some of the data in Agisoft Photoscan Pro, we wanted to test the pipeline in Agisoft to prove the 4D system could work and I think the results are pretty good. Unlike most commercial 4D Systems because of the multi camera array and the way the data is processed you get very little stretching or seaming, it’s far more “Volumetric”

8GB’s of data streamed in Softimage using Thiago Costa’s OBJ Sequencer with some extra ICE and Softimage trickery (best viewed at 720p)

Examples of me, trying to act tough ala Matrix style (performance is inspired by the research of George Borshukov for the Matrix films in 2003)

AGisoft “Frame-00124″ Test:

Synchronized Video Streams:

wmode=transparent[/youtube]

15 comments

  • lukas December 6, 2011

    hello. im following this blog for quite a while and im stunned, i love the latest agisoft scans. they came out great. did you also tried the free app called 123D catch ? formerly called project photofly ?
    i personally really like it. it does really nice and accurate results, its free and pretty quick to process (http://forums.cgsociety.org/showthread.php?f=43&t=1019262)

    Respond
    • Infinite December 6, 2011

      Hi and Thanks :) Yes I know of Photofly or whatever it is called now. Not a big fan, it doesn’t work as well with my Camera array as Agisoft does, it produces some weird circular clipping and blobbing effects. I also don’t like the idea of uploading my personal data to the cloud and the software is pretty closed off at the moment. If they can improve those thing and allow people to process offline I might look into it further.

      PS Your link didn’t work

      Respond
  • Oshri Even Zohar December 6, 2011

    Good work!!
    How far down the road are you to have it running in real-time?

    Respond
    • Infinite December 6, 2011

      Thanks! Artists are still very far down the road from real-time. So far they down the road, they haven’t even laid the cement yet! but with money anything is possible.

      For now, I will leave the real-time multi-million dollar development to the likes of Autodesk and Microsoft. The Kinect is as real-time as anyone will get for the next year or so, until the next Kinect 2.0 comes out. Microsoft know they are onto a good thing so I doubt they will put on the shelf big iterations on the tech but it will slowly get better. The above solution is costly and takes time.

      It’s incredible to be able to get even these kinds of results offline. It took the guys on the Matrix films $1,000,000′s and a large team of super talented people to create data like the above. Now 1 person can do it in a small studio, nearly. I just have to master the processing and re-targeting.

      Respond
  • Matt Chandler December 6, 2011

    Cool stuff Lee. Ive been figuring out a hacky, somewhat convulted way of getting the transform data to accompany batches of mesh deformation/capture like this with another 3rd party software used in combination. Drop me an email.

    Respond
    • Infinite December 6, 2011

      Many thanks. Will do now :)

      Respond
  • Karsten December 23, 2011

    Good work!
    Very interesting that Alexey processed your data in Agisoft Photoscan with such success, great (I use Photoscan too). For improving video quality (less compression) in Canon DSLR’s do you have tried Magic Lantern?

    http://magiclantern.wikia.com/wiki/Bit_rate

    Respond
    • Infinite December 23, 2011

      Hi Karsten, that is very interesting thanks for the link! I though the standard compression artifacts were the only option with Canons. Do you know how to use this Magic Lantern? I really want to try and minimize compression artifacts.

      Respond
  • Karsten December 24, 2011

    Hi Lee,
    a good point to start with Magic Lantern is http://magiclantern.wikia.com/wiki/Magic_Lantern_Firmware_Wiki .
    and how to install here http://magiclantern.wikia.com/wiki/550d_install .
    Good luck

    Karsten

    Respond
    • Infinite December 24, 2011

      Awesome! thanks : )

      Respond
  • Daniel January 8, 2012

    HI Lee,

    Love your work by the way. It’s very inspiring! I’m looking to do the same as you have done here but wanted to know where can I find the script to batch process the data from photoscan? I can’t seem to find it. Also is the script able to work in the standard edition or Pro? I don’t recall you mentioning which you were using. Thanks in advance and look forward to seeing more!

    Respond
    • Infinite January 8, 2012

      Hi Daniel, thanks.

      You will need the Pro version of Agisoft. I have the script here that Alexey wrote, although I am not sure I can share it just yet, I will have to confirm with Alexey first.

      Respond
  • Daniel January 9, 2012

    Hey Lee,
    Thanks for the reply. I totally understand that you need Alexey to say okay before giving it out. I appreciate that you are willing to help me out. It’s a very kind gesture and I thank you. I believe you have my email if your able to share the script. If not I thank you anyway for trying. Take care and keep up the great work!

    Respond
  • Vince Chmielewski January 12, 2012

    I noticed that your raw scanned mesh appears somewhat rough, whereas later images show it with much more detail. Have you used some kind of diffuse data from the images? Please elaborate if you can.

    Thanks,

    Vince

    Respond
    • Infinite January 12, 2012

      Yes the RAW scan is rough due to Passive capture. A high pass technique is used to apply high frequency details using a Push modifier, after a Smooth in Softimage on each sequenced Mesh.

      Lee

      Respond

Post a Reply

To submit your comment, click the image below where it asks you to...
Clickcha - The One-Click Captcha
©2012 IR-Entertainment Ltd. All Rights Reserved