VR system components VR - software, the missing piece?
Transcription
VR system components VR - software, the missing piece?
VR system components VR - software, the missing piece? n DISTRIBUTION/ COMMUNICATION HARDWARE A VR system (or game) can consist of one or more of the following items: n Visual display n Image generator n n n SIMULATION n INTERACTION n n 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software n n n n n n Physical structure Projectors Computer Peripherals n n n Device drivers Support tools Application n n ©Anders Backman, Dept. Computing Science, VR02 - Software 3 n Magnetic tracker 30-144Hz n A low report rate must not slow the application down. Availability n What happens if a trackersystem breaks down? n Application crash? Configuration VR - More than just graphics n n n Design is critical Reuse The use of design patterns can help getting a common language between implementors. 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 4 A lot of work to accomplish immersive VR int main(int argc, char **argv) { // Init // Loop forever and do: // Read devices VRdevice_read(); // Update objects VRupdate_objects(); // Render the Scene VRrender_scene(); } Software libraries from many vendors/projects could possible be involved n ©Anders Backman, Dept. Computing Science, VR02 - Software Software It is an integration of technologies n Different setups for different applications 2002-11-26 Why software? n Changing from one tracker to another may demand a total rewrite of the application when poorly done. Different report rates n 2002-11-26 2 Compatibility n Software: n ©Anders Backman, Dept. Computing Science, VR02 - Software Some integration issues Hardware: n Drivers 2002-11-26 Integration n Heterogenous, TCP, UDP, LAN, Modem,... Software n 1 Operating system, single multi CPU, shared/distributed memory... Network n HUMAN ISSUES SOFTWARE APPLICATIONS Gloves etc.. Computing environment n REALTIME ASPECTS Headspeaker, 2-4-8 speakers Interaction devices n GRAPHICS Camera based, magnetic, ultrasound, ... Sound system(s) n n One graphics card, Multi-pipe onyx, Cluster of PC:s Tracking system(s) n SOUND Computer scree, CAVE, Powerwall, ... n 5 Tools are needed 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 6 1 VE system architecture Software There are VR tools for lots of different areas: n graphics n Modeling n Translating n Authoring (events, …) n Display electronics n Virtual Environment Generation system head position hand position (Grapics software and workstation) Head- and handtracking electronics Immersive Visualization Facility n n Device support Rendering n Audio localizer n Distribution/Communication Sound n Simulation n 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software n 7 2002-11-26 Software - Modeling n n n n n n n n n n n Home-brewers way of life: n Great for modelling, easy to use No real control over polygons. Easy to write plugins to get full control No support for hiearcheal fileformat except VRML n n n n n n n Best for Rendering ©Anders Backman, Dept. Computing Science, VR02 - Software n 9 n n n 3D modeling tool (3D Studio, LightWave) Graphics (OpenGL), building a “graphic engine” Home written device drivers, if any. 2002-11-26 3D Studio/Maya Special tools for Special tools for Special tools for + Export plugin. lighting (generating Lightmaps) animations, scripting, ... everything. ©Anders Backman, Dept. Computing Science, VR02 - Software n Remember the whole picture DISTRIBUTION/ COMMUNICATION HARDWARE Low-level: API:s High-Level: GUI, drag-n-drop, Word for VR!! SIMULATION For prototyping a higher-level tool is often desirable Sometime low-level API give you the control you need over details Different tools are suitable for different purposes (special purpose tools) SOUND INTERACTION GRAPHICS n n n n n Very complicated to support all of these areas in an effective but still general way. The VR area are missing standards, work in progress. Result ⇒ Most VR authoring tools are still in its infancy, work in progress. Some nice systems exists though. Development cost is high n n 2002-11-26 10 Software Tools are in different levels n Hard to make a general product that lives up to all that. Game Developers way of life: Great at NURBS, although low control over polygons Software n 8 There are also tools that tries to match all of the above points. But: LightWave AC3D Blender … 2002-11-26 n ©Anders Backman, Dept. Computing Science, VR02 - Software n n Alias Maya n n Great for real-time modeling Full control over polygons! Hiearcheal file format. Rhino3D n n Handling of spatial sound connected to objects Simulation code for physical /artificial entities. Software 3D Studio n n Handling the consistency of a distributed DB. Multigen n n Communicates with the external VR hardware (trackers…) Graphics, usually a SceneGraph API on top of OpenGL n n Haptic system Converting file-formats Handling the actions in the application. n n Audio synthesizer Creating the 3D models ©Anders Backman, Dept. Computing Science, VR02 - Software 11 2002-11-26 Many platforms, high-end to low end Lot of hardware issues ©Anders Backman, Dept. Computing Science, VR02 - Software 12 2 Software - File-formats Software - Converters There are a lot of 3D file-formats out there Problems arise when trying to convert between them. Materials, textures disappear, ... n n n n VRML 1 .WRL n VRML 2 .WRL n n n n n n n n n n First standard, no events and such, only geometry n Second standard, contains a lot: interpolation, sound, key framing, ... No support for named objects n n Defacto standard for Vis-sim business. Hiearcheal format Supports links to other .flt files n n Contains no textures or materials AutoDesk .3DS n n n AutoDesk .DXF n There are a lot of converters out there too: n MultiGen’s OpenFlight .FLT n n n 3D studio mesh format. Geometry (mesh+textures) + animations. Wavefront .OBJ WorldToolKit Neutral File Format NFF Inventor .IV Every scenegraph API usually have its own native fileformat. n 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 13 2002-11-26 Software - Graphics n n n n n n n n n Industry standard for accessing the rendering hardware in an portable way. View models interactively in 3D space Control color and lightning manipulate pixels... Portable Full control In most cases to low level to start from scratch developing a whole immersive VR application n n n n n n n n n n n n n n n One of the first portable graphic API:s, now also OpenSource A mature multiplatform API. ©Anders Backman, Dept. Computing Science, VR02 - Software n 15 2002-11-26 Software - Graphics Support for 6DOF, Stereo, Collision detection, sound.. SGI/NT/Linux - Slow + Java = Portable, stable? VRML See OpenGL except for portability!! 2002-11-26 Aimed towards CAD industry Tessellation, optimization, Occlusion culling, multiprocessor support (SGI). NT, SGI (SUN, HP) Almost shutdown from SGI. Java3D (www.sun.com) OpenInventor n 14 Optimizer/Cosmo3D (www.sgi.com) Direct3D n n n OpenGL, n ©Anders Backman, Dept. Computing Science, VR02 - Software Software - Graphics Graphic APIs. n PolyTrans from Nugraf Crossroads (freeware) ivToflt, wrlToflt, … (SGI only) Some work, some don’t. (In some cases) 60% of work with a VR project can directly be related to fileconversion problems. Make sure you have a workflow ready. A well developed process of the whole chain of data, from generating it to rendering/simulating it. Not an API but still powerful Portable Performance? Outgrowing it self? X3D Mostly used as a fileformat. Not as a script language. ©Anders Backman, Dept. Computing Science, VR02 - Software 16 Software - Graphics § OpenScenegraph (www.openscenegraph.org) § A Core scenegraph n n § Handles the scene only, no windowing, user interaction, simulation etc... This is supposed to be handled by additional toolkits and libraries. Plib (www.plib.org) OpenSG (www.opensg.org) n n § Well object orientated, using Designpatterns. § Open source, get too peek at the source, AND to modify it. § Readers for most common formats, the community adds that. n n German research project Core scenegraph Multithreaded core design Gizmo3D (www.tooltech.com) n n § OpenFlight, 3DS, AC3D, VRML1, LWO, OBJ, GEO n n § Portable n Swedish company Mature Readers for most common filetypes Strictly OO Not OpenSource, but free for academic use. § Unix, Windows, MacOs X, Solaris, HP, (Playstation2) § Impostors n Performer (www.sgi.com) § First implementation in a scenegraph. n § Occlusion culling § -Under heavy development n n n § Things changes (interfaces, functionality) § Reaching 1.0, wich will mean a more stable core. n n § - Documentation n n § Code examples the best way to get into it. § Documentation is on the way. 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software n n 17 2002-11-26 Optimized for SGI (but now exists for Linux and Windows) A lot of functionality Multiprocessing System stress and load management Asynchronous database paging Morphing Clip Mapping for handling of really large textures. Fixed-frame-rate capability DVR (Dynamic Video Resolution) (On SGI Onyx) Not really easy to work with. ©Anders Backman, Dept. Computing Science, VR02 - Software 18 3 Software - Hardware? n Software Device drivers n n n n n A VR toolkit should contain most of the following issues If you choose to program from scratch in Performer, OpenGL, ... for the graphics, you usually have to communicate with the hardware yourself. Different trackers have different ways of communication, different data-structures, … Makes it hard to write yourself To use finished products help a lot. n Real time graphics n Importing models, filters n Model libraries n Animation n Collision detection n n n n n 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 19 n n n Networking n n n n n n n n Support for trackers, gloves, ... n n Exchanging minimal level of data over a network so several users can cooperate in a VE. Dead reckogning algorithms Interest groups Support for different projection systems ©Anders Backman, Dept. Computing Science, VR02 - Software 21 2002-11-26 VR Juggler n n n n n n n n n n n n n 22 Commercial n IDE/API: Multigen, SmartScene, VEGA Prime n VEGA is one of the most competent VR development toolkits available. Lots of different modules for: n n n Write on a PC running windows, using mouse for interaction Later run on a multipipe onyx with trackersystem n n Support for cluster rendering Sound support Does not include a scenegraph, depends on either OpenGL, Performer, OSG, OpenSG Sounds wonderful, wheres the catch? n ©Anders Backman, Dept. Computing Science, VR02 - Software Software University of Iowa Open source Developed to make code once, compile and run anywhere for VR. Incorporates hardware support with display support. GUI:s for setting up the environment. You dont have to decide the interaction hardware while writing the code. This is configurated later on. n CAVElib, Commercial Maverick Open Source, AIG, UK DIVE, SICS, Sweden. Simple Virtual Environment (SVE), Georgia Tech, USA. MR Toolkit, University of Alberta, Canada DIVERSE, Open Source (built on Performer) VR Juggler, Open Source, Iowa State University Stereo Viewing n n 20 Research Inverse kinematics, rigid-body-dynamics, ... n Bounding box, geometry ©Anders Backman, Dept. Computing Science, VR02 - Software n Sensors Interpolators, movie-clips, ... Listing of VR software Controlling 3D sound sources and listeners, loading of different sound files. n n n Simulation n 2002-11-26 Where actions are programmed without need for compiling, etc. Trigger for actions. Audio n Primitives: box, polygons, spheres, cones, ... Software Scripting language n CAD models can be surface patches or NURBS or CSG (Constructive Solid Geometry) 2002-11-26 Software n Level-of-detail, culling, ... n Marine simulation - Dynamic ocean simulation Radar simulation Special effects - Explosions, smoke, fire, missile trail… Terrain generation Sounds a lot like the US Defence to me? Expensive Large Monumental software Many requirements of tools/versions for getting it compiled. 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 23 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 24 4 Software n Sense8 n n n n n API: WorldToolKit IDE: WorldUp World2World IDE: WorldUp from Sense8 Integrated Development Environment n Eon Reality’s EON Superscape VR n Muse Technologies Inc. n n n n Gone out of business. Gone out of business Dvise n n Software Not developed anymore. Application development Interfaces n n n 2002-11-26 Boston Dynamics Inc. DI guys human simulation Magma from ReachIn Technologies ©Anders Backman, Dept. Computing Science, VR02 - Software 25 2002-11-26 Software n n n 2002-11-26 n Used for ergonomic evaluations Developed at University of Pennsylvania Has 74 segments, 73 joints, 22 segment spine, 150DOF Can be customized based on scaling, obeys joint and strength limits taken from NASA studies, TJ:s dimensions is based on 132 anthropometric studies ©Anders Backman, Dept. Computing Science, VR02 - Software n n 27 dVise from Parametric Technology Corp (former Division) Suitable for CAD prototyping Video 2002-11-26 Software n n 28 Simulation driven or graphic driven? n Init graphics() SetupScene() SetupDevices() SetupActions() while(1) do readDevices() callActions() updateModels() renderGraphics() enddo ©Anders Backman, Dept. Computing Science, VR02 - Software ©Anders Backman, Dept. Computing Science, VR02 - Software Software A general VR application 2002-11-26 26 Software IDE: EAI’s Transom Jack n ©Anders Backman, Dept. Computing Science, VR02 - Software n Usually the simulation has been pushed away for the graphics. This is undergoing a change. For example the games requires more and more precise simulation underneath. AI, physics, … n n This is getting more and more important. It always depends on what you are looking for. n n 29 2002-11-26 In an Immersive Visualization system where people is trying to find pockets of oil from measurements of the earth’s soil, maybe an authentic simulation of the friction is not of primarily interest. For a vehicle simulator, where the vehicle’s interaction with the terrain is important, it is obvious that some sort of physical simulation is needed. ©Anders Backman, Dept. Computing Science, VR02 - Software 30 5 Software n Software A graphic driven simulation will prioritize the graphics. The attributes of the graphic entity is given to the simulation n n Graphic representation (scene graph) Simulation representation A B Simulation code for C If User enter C then Do Action C C D Whereas a simulation driven simulation is based on the entities in the simulation. Rather then building the graphical scene and than add dynamic behaviour, there is a simulation which contain all the information needed to simulate and generate the visible parts of the object. Mass, inertia, speed, ... E Mass, inertia, speed, ... Simulation code for E In each timeframe check for contact with D (ground) 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 31 2002-11-26 Software n Mass, inertia, speed, ... Graphics ©Anders Backman, Dept. Computing Science, VR02 - Software 32 Software Some software packages exists with behavioral built into it. n Generates n Vortex n Havok EON - Contains springs and multibody n n A virtual tour of Tutankhamuns tomb. n n A Walk around a town square Physics SDK Plugins for 3D Studio Modelica n n Reality Training™ example, where the user is practicing odontological techniques on a virtual patient, utilizing a force-feedback device. Physics SDK A research project in Linköping. Generic language for simulations. n n Rigid body, fluid, electronics, ... Transom Jack, lots of inverse kinematics and simulation around the soft tissues in the body. Reality Training™ example, where the user is practicing to dismantle a car brake disc. 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 33 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 34 And not to mention - the WEB n n 3D chat/model builders Real-time multi-user interaction in 3D environments n n n n n n n n Active Worlds Adobe Atmosphere Blaxxun Ultimate 3D Chat TalkWorld Bang V-Chat Deep Matrix 2002-11-26 ©Anders Backman, Dept. Computing Science, VR02 - Software 35 6