Stereophotogrammetry and 3D printing:Critical Practices and Intellectual Property
Owen Mundy, Fulbright Visiting Professor, University of Klagenfurt, Austria

Owen Mundy, Stereophotogrammetry and 3D printing:Critical Practices and Intellectual Property, in Andrea Wallace and Ronan Deazley, eds, Display At Your Own Risk: An experimental exhibition of digital cultural heritage, 2016.

---

 

INTRODUCTION

From photography to printable 3D objects, technology continues to provide new methods of creative production which complicate authorship and copyright norms. Thanks to developments in stereophotogrammetry and 3D printing, it is possible to replicate an object using only photographs of the original work. In three recent examples artists have used this process to copy works and cultural artefacts protected by copyright or patents. One collaborative group claims to have covertly scanned and printed a 3D copy of the famous bust of Nefertiti within the Neues Museum in Berlin, while another recreated Marcel Duchamp’s famous chess set from archive photographs. In a third example, a collaborative team developed a collection of adapter bricks enabling the interconnection of popular children’s construction toys. In each instance the artists released their digital models on the internet so that anyone could download and print their own copies. While cultural institutions and corporations like Google have privileged access to digitize and distribute images and digital models of artworks, the artists’ actions provoked at least one institution to respond with legal threats. These works raise important questions about the scope, nature, and function of intellectual property laws. Does the copyright regime incentivize and enable the realization of innovative works and ideas, or has the influence of industry lobbynomics resulted in a copyright landscape that prioritizes corporate interests over those of individual creators and creative practices.

STEREOPHOTOGRAMMETRY, COPYRIGHT, AND GOOGLE

This essay starts with a discussion of some of the technical underpinnings of the processes to digitize 3D objects and space which, like photography and other media, are increasingly accessible to artists and hobbyists. Photogrammetry is a process which evaluates photographs to measure surfaces and physical objects. It has been in use since photography was invented and has many scientific, engineering, and mapping applications. In addition to the geospatial uses, the indexing of three-dimensional points of close-range objects and spaces, or stereophotogrammetry, has been employed widely in cinema, gaming, and other storytelling forms. In this process multiple cameras record a subject from many different angles and the footage is stitched together using special software. The result is a photorealistic scene that mimics the spatiality of real life.

In the Matrix films for example, stereophotogrammetry is combined with other special effects to show action from multiple perspectives in slow motion sequences (Fig. 1). The producers first created a digital 3D model using photographs from a scene. Then they used a time slice rig which contains many cameras that each film simultaneously from different angles around the action. Finally, they matched the camera angle in the film to the camera path of the time slice rig and layered the 3D model over the green screen background, allowing them to pan 360 degrees around the action in slow motion.^[1][2] The effect reinforced the importance of the protagonist, Neo or ‘The One,’ to the storyline of the film by creating an illusion that he is absolutely in control of his actions in multiple dimensions of time and space.^[3]

The use of photographs to index three-dimensional spaces and objects in the real world is the basis for many other such illusions in digital spaces. Emerging from early and awkward ‘CAVE’ times,^[4] virtual reality (VR) is enjoying a resurgence thanks to advances in stereophotogrammetry, the ubiquity of game developers and frameworks, and the introduction of affordable VR headsets. The availability of the headsets, which create an illusion of depth using stereoscopy (displaying two images viewed from slightly different angles to match the binocular vision of human eyes) when viewing digital 3D spaces, has had the greatest impact on the means of production, resulting in new VR experiences in everything from gaming, to interactive documentary, to pornography.

Rather than stitching the images together surrounding a virtual space, the Google Earth application creates an interactive experience using millions of satellite photographs tiled together onto a virtual object. As users zoom into representations of the Earth, the moon, or Mars, more detailed tiles are downloaded and placed on the sphere’s surface determined by the satellite camera’s position, altitude, and camera angle. The Google Street View camera system, which can be mounted on cars, bikes, boats, snowmobiles, camelback,^[5] or humans using a special backpack,^[6] is the ultimate land-based stereophotogrammetry system. Equipped with 15 cameras and a range of sensors including GPS (Global Positioning System), wheel speed sensors, and laser range scanners, it can be used to photograph and index any site on earth.

Recent partnerships between Google and 17 top cultural institutions around the world now find the Street View technology being employed within the interiors of museums like The Louvre and the British Museum (Fig. 2).^[7] Similar to the Google Books effort, which attempted to scan and convert to searchable text all books in the world under the erroneous pledge of cultural benefit, Google Cultural Institute undertakes scanning the artwork and interiors of museums to make cultural material ‘accessible to anyone, anywhere.’^[8] This openness rhetoric aligns with other utopian visions promised by Silicon Valley which play down the profit motive and discount problems their technology or actions may cause. For example, undertaking a digitization project of this nature triggers various complex copyright issues, and particularly as many institutions collect and show the work of living artists, or works that are not yet in the public domain. Also, when dealing with collections of objects curated from around the world it is important to be aware that each country has distinct laws governing intellectual property, and each may have to be considered. And of course, each cultural institution will have their own attitude to and policy on the reproduction of works in their collection.

However, unlike the Books project,^[9] Google Cultural Institute has apparently taken steps to preempt lawsuits. In addition to espousing the cultural benefit of their work, they’ve partnered with the relevant institutes, obtained permission from copyright owners, and notably have invested into a research lab in Paris to inspire and create connections between art, culture, and technology. Google’s service to culture should be noted, but their goal is most likely to further secure dominance in all ways that information is consumed online. Like their other ‘free’ products, the Google Art Project introduces new ways to mine and assess data about human activities for a company that reported recently that 90% of their $74.5 billion revenue came from advertising in 2015.^[10][11] It is also potentially a part of their European project that, like their funding of the lab in Paris and the Institute for Internet and Society at Humboldt University in Berlin, is intended to sway public opinion in Europe towards sympathy for the software giant given recent debates over data privacy and whether or not Google’s practices are monopolistic.^[12]

Of course, Google can access and digitize these spaces and objects because Google is a company with very deep pockets. Unlike Google, an ordinary museum visitor will enjoy little to no bargaining power when dealing with an institution, and will be bound by whichever photography policies the institute has in place. These policies nearly always prohibit the visitor from using his or her photographs for commercial purposes regardless of whether the photo was taken using a smartphone or professional grade equipment. For example, using equipment like a tripod in the British Museum requires special permission and the images are restricted to noncommercial use.^[13] Other museums might go so far as to ban visitor photography,  or even sketching. (Fig. 3).^[14] To be sure, Google’s position is a privileged one.

ART, ARTISTS AND 3D PRINTING

Consider the experience of two non-corporate creators making use of stereophotogrammetry. Inspired by Marcel Duchamp’s famous chess set, artists Scott Kildall and Bryan Cera used photographs to digitally reconstruct the game as a 3D printable model (Fig. 4). In their view, their use of archival photos of Duchamp’s hand-carved game resurrected the work and revived interest in the artist. When they shared their process and uploaded the digital files allowing anyone to download and print the model they felt it a conceptual gesture which acknowledged Duchamp’s own art practice of the readymade, which was itself an act of appropriation and recontextualization.^[15]

Since Duchamp created the chess set before 1923, in the United States the work is in the public domain. However, that does not mean the work is in the public domain everywhere in the world. The complexities of the international copyright regime are discussed elsewhere in this collection^.[16] For our purposes, what is important is that the Duchamp Foundation disagreed with Kildall and Cera. The Foundation claimed the reconstructed chess set violated Duchamp’s copyright in France (according to the standard European copyright term of the life of the author plus 70 years).^[17] In their cease and desist letter to the artists the Foundation stated the reconstruction was an illegal adaptation and that, even if no money was collected through the distribution of the files online, under French law they may be liable for substantial penalties and even imprisonment.

If the issue was to be determined solely by US law, the artists could rely on a number of strong defences including the fact that the original was created in 1918 (and so was out of copyright) as well as various arguments concerning lawful fair use. Ultimately, their work was a creative homage to Duchamp, intended to spark conversation about Duchamp’s legacy and practice in the context of the information age. However, faced with potentially hefty court costs and a possible prison sentence, the artists gave in to the Foundation’s demands and removed the digital files from the internet. Whether or not this outcome was actually in the best interests of the Foundation and their mission to further establish appreciation of the work of Duchamp is open to debate. But it demonstrates the difficulties – legal, financial, and otherwise – that individual creators face when they are interested in re-imagining our existing cultural heritage in new, innovative, and provocative ways. Had Google proposed this idea to the Duchamp Foundation, the result might well have been different.

The Duchamp Chess Set also shows how reproduction, modification, and distribution are deeply rooted in contemporary cultural practice, and a misunderstanding of these phenomena can actually do more harm than good for organizations and rightsholders like the Duchamp Foundation. This issue has even greater implications when considering the rise in the use of 3D printing for fan art. ‘The technology is coming whether we like it or not,’ says Michael Weinberg, a lawyer at Public Knowledge, an organization which advocates for an open Internet. Rather than using resources to attack individuals and websites with takedown notices for sharing copies of such works, he encourages copyright owners to learn to embrace the technology and fans who use it.^[18]

Unlike ‘ripping’ music or films from physical media to a computer, the process Kildall and Cera used to digitize the physical object is neither automatic nor instantaneous. The artists used photographs to ‘trace’ the chess set by hand, manually locating points in the image on screen in order to maintain proportionality. It’s a technical and creative process in its own right which requires both meticulous attention to detail and knowledge of the medium. This is done using tools like Autodesk’s 123D, Rhino3D, or various other CAD (Computer Aided Design) software.

When Golan Levin and Shawn Sims created the Free Universal Construction Kit (Fig. 5) they used a process similar to the above, drafting the objects on the computer with measurements taken by hand with a micrometer and from light projected by an optical comparator. The kit, which was developed jointly at the Studio for Creative Inquiry, includes nearly 80 adapter bricks to allow intersection between popular children’s construction toys and is meant to encourage ‘new forms of dialog between otherwise closed systems – enabling radically hybrid constructive play.’^[19]

The artists’ viewpoint is that with all the proprietary building block systems available a public service is needed to meet the needs of the children (or adults) who want to think and build based on their creative limits, not the constraints of the various patents and commercial interests at work. The Free Universal Construction Kit includes adapters between Lego, Duplo, Fischertechnik, Gears! Gears! Gears!, K’Nex, Krinkles (Bristle Blocks), Lincoln Logs, Tinkertoys, Zome, and Zoob. These adapters can be freely downloaded as a set of 3D models in .STL format, for reproduction on any 3D printer.

As a general rule, while creative works such as art, literature, and music are protected by copyright, objects which perform a function or are useful in some way, like interlocking bricks, are covered by other forms of intellectual property, such as design and patent law. The philosophy behind protection for patents is similar to that for creative works – protecting commercial interests in order to encourage the development of new ideas, while at the same time setting a defined term of protection so that, at some point, the work is released back to society for reuse, development, and improvement. Unlike the lengthy terms of protection that copyright works enjoy, patents are typically protected for no longer than 20 years. When Levin and Sims created their Free Universal Construction Kit they deliberately avoided possible claims of patent infringement by only releasing adapters for older systems first. For eight of the ten building systems that feature in their work, relevant patents have already expired; the final two, Zoob and ZomeTool, will be released in 2016 and 2022 respectively.^[20]

This subversive yet (literally) playful action is intended to overcome limitations introduced by mass-produced culture. It smartly walks the line between fine art, design innovation, and creative disturbance by combining a thoughtful and provoking rhetoric with a utilitarian system to challenge passive consumerism and allow the interconnection of toys which previously couldn’t fit together. It shows that while 3D printing is still very new to the world it offers ‘the same disruptive potential as the original printing press.’ In the same way ‘moveable type spread across Europe and democratized knowledge, the proliferation of 3D printers eventually promises to democratize creation.’^[21]

The two examples discussed above evidence only one of many ways to translate real objects into digital 3D representations. More automated processes like 3D scanning take many forms. Some scanners actually make contact with the objects they scan, probing the surface and collecting measurements, while non-contact based scanners collect information about a subject’s surface and volume using lasers or other radiation projected at the object.

The more accessible and therefore potentially copyright-infringing methods use a process similar to the photogrammetric procedure described above. Non-contact passive light scanning, which could be performed with very specialized equipment or even a smartphone, use ambient light to construct 3D objects. Like Google Earth or time slice rigs used in films, they make multiple photographs around the subject, stitching them together by linking recognizable points on the surface to create a three-dimensional point cloud. Taking this process a step further, the point cloud can be converted into polygons and cleaned up in a way that, if done with high enough precision, can be output to a CNC (Computer Numerical Control) machine such as a 3D printer or laser cutter.

This is the method that artists Nora Al-Badri and Jan Nikolai Nelles claimed to use to covertly make a 3D scan of the famous bust of Queen Nefertiti inside the Neues Museum in Berlin (Fig. 6). The bust of Nefertiti is a 3,300-year old work of art found in Egypt by German archaeologists in 1912. Controversy has surrounded the bust since it was found and brought to Germany, with Egypt claiming it had been taken illegally, and the museum refusing its return.^[22] In addition to releasing the printable 3D files online for free^,[23] the artists created an exact copy of the original in polymer resin which will reside permanently in the American University in Cairo in place of the actual bust (Fig. 7).^[24]

The Neues Museum’s visitor policy prohibits photography, yet the artists claim they were able to make enough images using what appears to be a Microsoft Kinect in the video they released. This statement is not without its own controversy, as experts who have analyzed the artists’ file state it is very detailed, containing over nine million polygons, and bears a striking resemblance to the museum’s own digital scan of the bust. Whether or not they actually created the file themselves, the artists’ effort to ‘repatriate’ the bust, while sharing its digital copy online, has proved highly effective in provoking difficult questions regarding the history of the collection of disputed artefacts by western museums and private collections.

CONCLUSION

Cultural institutions are either embracing or ignoring the changes brought about by the information age. While some organizations partner with Google, or hold ‘scanathons’ to digitize and release scans of huge numbers of their objects into the public domain (like The Metropolitan did in collaboration with Thingiverse.com), others appear to be significantly behind the curve. This is important because as society continues the turn towards a software-based culture where what exists is only that which is accessible and freely shared, those who maintain closed systems and prohibit appreciation of their cultural property will find fewer and fewer audiences to patronize, support, and equally contribute their appreciation to new audiences. Unlike the increased commodification of culture by corporations with vast legal teams, museums and cultural institutions should be the first to realize that the more accessible and understandable they make their collections, the more likely they will grow and become a central site for cultural understanding.

 

^[1]          ‘Virtual Reality, HDR, Photogrammetry at ICT’ (2008), available at: https://www.youtube.com/watch?v=UUvAVjUnE8M (accessed: 23 April 2016)

^[2]          ‘The Matrix Behind The Scenes – Rooftop (1999) – Keanu Reeves Movie HD’ (2014), available at: https://www.youtube.com/watch?v=Kjcv-JtUOgA (accessed: 23 April 2016).

^[3]          ‘Zo zien de scenes van bekende films en series er met en zonder special effects uit’, n.d., available at: http://www.bytez.nl/zo-zien-de-scenes-van-bekende-films-en-series-er-met-en-zonder-special-effects-uit/ (accessed: 23 April 2016).

^[4]          ‘The Rise and Fall and Rise of Virtual Reality’, n.d., available at: http://www.theverge.com/a/virtual-reality (accessed: 23 April 2016).

^[5]          ‘Pakai Unta, Google Petakan Padang Pasir Kamis’ (9 October 2014), available at: http://wartakota.tribunnews.com/2014/10/09/pakai-unta-google-petakan-padang-pasir (accessed: 23 April 2016).

^[6]          ‘Google Street View scales Snowdon using camera backpacks’ (11 December 2015), available at: http://www.bbc.com/news/uk-wales-north-west-wales-35064087 (accessed: 23 April 2016).

^[7]          Collett-White, M., ‘Google Offers Virtual Tours of 17 of the Top Museums Using Street View Technology’, n.d., available at: http://artdaily.com/news/44628/Google-Offers-Virtual-Tours-of-17-of-the-Top-Museums-Using-Street-View-Technology (accessed: 23 April 2016).

^[8]          Google Cultural Institute website, n.d., available at: https://www.google.com/culturalinstitute/about/ (accessed: 23 April 2016).

^[9]          Meyer, R., ‘After 10 Years, Google Books Is Legal’ (20 October 2015), available at: http://www.theatlantic.com/technology/archive/2015/10/fair-use-transformative-leval-google-books/411058 (accessed: 23 April 2016).

^[10]         ‘Distribution of Google’s revenues from 2001 to 2015, by source’, n.d., available at: http://www.statista.com/statistics/266471/distribution-of-googles-revenues-by-source/ (accessed: 23 April 2016).

^[11]         ‘Annual revenue of Google from 2002 to 2015 (in billion U.S. dollars)’, n.d., available at: http://www.statista.com/statistics/266206/googles-annual-global-revenue/ (accessed: 23 April 2016).

^[12]         ‘Independent Research Institute for Internet and Society starts with four partners’ (2011), available at: https://www.hu-berlin.de/de/pr/pressemitteilungen/pm1107/pm_110711_01 (accessed: 23 April 2016).

^[13]         The British Museum, ‘Photography: Taking photographs in the museum’, n.d., available at: http://www.britishmuseum.org/about_us/services/photography.aspx (accessed: 23 April 2016).

^[14]         Wainwright, O., ‘No sketching’: V&A signs betray everything the museum stands for’ (22 April 2016), available at: http://www.theguardian.com/artanddesign/2016/apr/22/va-museum-no-sketching-signs-draconian (accessed: 24 April 2016).

^[15]         Noe, R., ‘Combining Detective Work, CAD, and 3D Printing to Recreate Duchamp’s Lost Chess Set from 1918’ (9 July 2014), available at: http://www.core77.com/posts/27268/Combining-Detective-Work-CAD-and-3D-Printing-to-Recreate-Duchamps-Lost-Chess-Set-from-1918 (accessed: 23 April 2016).

^[16]         Torremans, P., ‘Copyright infringement, exceptions and limitations and access to shared cultural heritage across borders.’

^[17]         Norton, Q., ‘The International Fight Over Marcel Duchamp’s Chess Set’ (8 September 2015), available at: http://www.theatlantic.com/technology/archive/2015/09/the-international-fight-over-marcel-duchamps-chess-set/404248/ (accessed: 23 April 2016).

^[18]         Henn, S., ‘As 3-D Printing Becomes More Accessible, Copyright Questions Arise’ (19 February 2013), available at: http://www.npr.org/sections/alltechconsidered/2013/02/19/171912826/as-3-d-printing-become-more-accessible-copyright-questions-arise (accessed: 23 April 2016).

^[19]         Levin, G. and S. Sims, ‘Free Universal Construction Kit’ (2012), available at: http://studioforcreativeinquiry.org/projects/free-universal-construction-kit (accessed: 23 April 2016).

^[20]         Free Art and Technology [F.A.T.] Lab and Sy-Lab, ‘The Free Universal Construction Kit’ (20 March 2012), available at: http://fffff.at/free-universal-construction-kit/ (accessed: 23 April 2016).

^[21]         Hanna, P., ‘The next Napster? Copyright questions as 3D printing comes of age’ (6 April 2011), available at: http://arstechnica.com/tech-policy/2011/04/the-next-napster-copyright-questions-as-3D-printing-comes-of-age/ (accessed: 23 April 2016).

^[22]         Watson, R., ‘Is the “Stolen” 3D Scan of Nefertiti Actually a Larger Hoax?’ (17 March 2016), available at: http://skepchick.org/2016/03/is-the-stolen-3D-scan-of-nefertiti-actually-a-larger-hoax/ (accessed: 23 April 2016).

^[23]         Nelles, J.N. and N. Al-Badri, ‘Nefertiti Hack’, n.d., available at: http://nefertitihack.alloversky.com/ (accessed: 23 April 2016).

^[24]         Voon, C., ‘Artists Covertly Scan Bust of Nefertiti and Release the Data for Free Online’ (19 February 2016), available at: http://hyperallergic.com/274635/artists-covertly-scan-bust-of-nefertiti-and-release-the-data-for-free-online/ (accessed: 23 April 2016).

 

---

Works Cited

‘Annual revenue of Google from 2002 to 2015 (in billion U.S. dollars)’ n.d., available at: http://www.statista.com/statistics/266206/googles-annual-global-revenue/

‘Distribution of Google’s revenues from 2001 to 2015, by source’ n.d., available at: http://www.statista.com/statistics/266471/distribution-of-googles-revenues-by-source/

‘Google Street View scales Snowdon using camera backpacks’ (11 December 2015), available at: http://www.bbc.com/news/uk-wales-north-west-wales-35064087

‘Independent Research Institute for Internet and Society starts with four partners’ (2011), available at: https://www.hu-berlin.de/de/pr/pressemitteilungen/pm1107/pm_110711_0

‘Pakai Unta, Google Petakan Padang Pasir Kamis’ (9 October 2014), available at: http://wartakota.tribunnews.com/2014/10/09/pakai-unta-google-petakan-padang-pasir

‘The Matrix Behind The Scenes – Rooftop (1999) – Keanu Reeves Movie HD’ (2014), available at: https://www.youtube.com/watch?v=Kjcv-JtUOgA

‘The Rise and Fall and Rise of Virtual Reality’ n.d., available at: http://www.theverge.com/a/virtual-reality

‘Virtual Reality, HDR, Photogrammetry at ICT’ (2008), available at: https://www.youtube.com/watch?v=UUvAVjUnE8M

‘Zo zien de scenes van bekende films en series er met en zonder special effects uit’ n.d., available at: http://www.bytez.nl/zo-zien-de-scenes-van-bekende-films-en-series-er-met-en-zonder-special-effects-uit/

British Museum, ‘Photography: Taking photographs in the museum’ n.d., available at: http://www.britishmuseum.org/about_us/services/photography.aspx

Collett-White, M., ‘Google Offers Virtual Tours of 17 of the Top Museums Using Street View Technology’ n.d., available at: http://artdaily.com/news/44628/Google-Offers-Virtual-Tours-of-17-of-the-Top-Museums-Using-Street-View-Technology

Free Art and Technology [F.A.T.] Lab and Sy-Lab, ‘The Free Universal Construction Kit’ (20 march 2012), available at: http://fffff.at/free-universal-construction-kit/

Google Cultural Institute website n.d., available at: https://www.google.com/culturalinstitute/about/

Hanna, P., ‘The next Napster? Copyright questions as 3D printing comes of age’ (6 April 2011), available at: http://arstechnica.com/tech-policy/2011/04/the-next-napster-copyright-questions-as-3D-printing-comes-of-age/

Henn, S., ‘As 3-D Printing Becomes More Accessible, Copyright Questions Arise’ (19 February 2013), available at: http://www.npr.org/sections/alltechconsidered/2013/02/19/171912826/as-3-d-printing-become-more-accessible-copyright-questions-arise

Levin, G. and S. Sims, ‘Free Universal Construction Kit’ (2012), available at: http://studioforcreativeinquiry.org/projects/free-universal-construction-kit

Meyer, R., ‘After 10 Years, Google Books Is Legal’ (20 October 2015), available at: http://www.theatlantic.com/technology/archive/2015/10/fair-use-transformative-leval-google-books/411058

Nelles, J.N. and N. Al-Badri, ‘Nefertiti Hack’ n.d., available at: http://nefertitihack.alloversky.com/

Noe, R., ‘Combining Detective Work, CAD, and 3D Printing to Recreate Duchamp’s Lost Chess Set from 1918’ (9 July 2014), available at: http://www.core77.com/posts/27268/Combining-Detective-Work-CAD-and-3D-Printing-to-Recreate-Duchamps-Lost-Chess-Set-from-1918

Norton, Q., ‘The International Fight Over Marcel Duchamp’s Chess Set’ (8 September 2015), available at: http://www.theatlantic.com/technology/archive/2015/09/the-international-fight-over-marcel-duchamps-chess-set/404248/

Torremans, P., ‘Copyright infringement, exceptions and limitations and access to shared cultural heritage across borders,’ in A.Wallace and R. Deazley, eds, Display At Your Own Risk: An experimental exhibition of digital cultural heritage (2016)

Voon, C., ‘Artists Covertly Scan Bust of Nefertiti and Release the Data for Free Online’ (19 February 2016), available at: http://hyperallergic.com/274635/artists-covertly-scan-bust-of-nefertiti-and-release-the-data-for-free-online/

Wainwright, O., ‘No sketching’: V&A signs betray everything the museum stands for’ (22 April 2016), available at: http://www.theguardian.com/artanddesign/2016/apr/22/va-museum-no-sketching-signs-draconian (accessed: 24 April 2016).

Watson, R., ‘Is the “Stolen” 3D Scan of Nefertiti Actually a Larger Hoax?’ (17 March 2016), available at: http://skepchick.org/2016/03/is-the-stolen-3D-scan-of-nefertiti-actually-a-larger-hoax/