
Here you can find an expanded proposal laying out the plans more methodically.
Please scroll below /////////////////////////////////////////////////////////////////////////////////////////////
2
Jump to a section here
3
Photogrammetry and Gaussian splatting teaching pathways
1
Yes, adding photogrammetry and Gaussian splatting technologies to your curriculum will enhance students' technical skillsets in 3D scanning, reconstruction, and rendering, especially for game design and virtual production. Both technologies play crucial roles in creating highly realistic environments and assets by leveraging real-world data, which can drastically reduce the time and effort needed to create intricate 3D models from scratch.
Here’s how to integrate these technologies into your curriculum, alongside virtual production and game design elements.
Adding Photogrammetry and Gaussian Splatting to the Curriculum
1. Introduction to Photogrammetry
-
What is Photogrammetry? Begin by introducing students to photogrammetry, a technique for creating 3D models from 2D photographs. This technology allows for highly detailed and accurate 3D reconstructions of real-world objects, environments, or characters, which can be used in both games and virtual production.
-
Applications in Industry: Discuss the use of photogrammetry in industries such as gaming (e.g., Battlefield series), film (e.g., The Mandalorian), and architecture. Show how it allows developers to create realistic game environments and virtual sets by scanning real-world locations or props.
2. Photogrammetry Workflow
-
Capturing High-Quality Images: Teach students the basics of capturing images for photogrammetry, covering camera settings, lighting conditions, and techniques for achieving accurate coverage (e.g., overlap and angles). They will learn how to create digital assets from a wide variety of sources, including objects, architecture, and terrain.
-
Photogrammetry Software: Introduce students to photogrammetry software such as RealityCapture, Agisoft Metashape, or Meshroom. These tools help convert the photos into 3D models.
-
Model Optimization: Once the 3D models are created, students should learn to optimize them for use in game engines or virtual production by reducing poly count, cleaning up meshes, and creating textures. This step includes baking high-resolution textures onto low-poly models for efficient real-time rendering.
-
Use in Unreal Engine and Unity: Teach students how to import their photogrammetry-generated models into game engines like Unreal Engine or Unity. This allows them to integrate these assets into games or virtual environments seamlessly.
3. Practical Projects Using Photogrammetry
-
Building Game Environments: Students can use photogrammetry to scan real-world locations (such as forests, urban areas, or historical sites) and create game environments or virtual production sets.
-
Asset Creation: Assign projects where students scan everyday objects (e.g., furniture, rocks, vehicles) and create highly detailed assets for use in games or virtual productions.
-
Virtual Set Design for Film: Have students create virtual film sets by scanning real-world locations and combining them with green screen footage or motion capture, blending photorealism with storytelling in virtual production.
4. Introduction to Gaussian Splatting
-
What is Gaussian Splatting? Introduce Gaussian splatting as a novel technique for rendering 3D scenes using point clouds rather than traditional polygons. Gaussian splatting helps to efficiently create realistic 3D environments by applying Gaussian distributions to the point cloud data, leading to faster rendering with fewer computational resources.
-
Applications in Industry: Highlight how Gaussian splatting is being used in cutting-edge real-time rendering systems, particularly for creating large, detailed outdoor environments or dense forest scenes. It’s especially relevant in projects where high performance and photorealism are required, such as open-world games or large-scale virtual production sets.
5. Gaussian Splatting Workflow
-
Generating Point Clouds: Show students how to generate point clouds from either photogrammetry data or LiDAR scans (Light Detection and Ranging). Students will learn how to convert these point clouds into assets that can be used in game engines.
-
Implementing Gaussian Splatting: Teach students to use Gaussian splatting to convert dense point clouds into smooth, rendered surfaces in real-time, without the need for traditional polygon-based modeling. Tools like Kaolin (by NVIDIA) or custom implementations within Unreal Engine can be explored for this purpose.
-
Optimization for Game Engines: Instruct students on how to optimize Gaussian splats for real-time rendering, ensuring that scenes can run smoothly even on lower-spec hardware, a critical skill for game design and virtual production.
6. Practical Projects Using Gaussian Splatting
-
Environment Creation: Have students create dense, highly detailed natural environments (e.g., forests, mountains, underwater scenes) using photogrammetry and Gaussian splatting. This allows for extremely realistic terrains and vegetation that can be rendered efficiently in real-time.
-
VR and AR Applications: Gaussian splatting is ideal for use in VR and AR because it enables faster rendering of high-quality visuals with minimal performance overhead. Students can develop VR or AR environments that make use of this technology for virtual production or interactive experiences.
-
Interactive Virtual Productions: In virtual production projects, students can use Gaussian splatting to create expansive environments that blend seamlessly with real-time live-action footage, particularly in outdoor or large-scale scenes.
Integrating Photogrammetry and Gaussian Splatting into the Virtual Production Workflow
1. Photogrammetry and Gaussian Splatting in Virtual Set Design
-
Photorealistic Virtual Sets: Students can design entire film sets using photogrammetry to scan real-world locations and objects, importing the data into Unreal Engine or Unity for virtual production. Gaussian splatting can be applied to render these environments quickly and efficiently during live filming.
-
Efficient Real-Time Rendering: Teach students how to use Gaussian splatting to reduce computational load during real-time filming, allowing for smoother workflows in virtual production where time and resources are constrained.
-
Set Extension: Demonstrate how photogrammetry and Gaussian splatting can be used to extend real-world sets with virtual elements, seamlessly blending physical and digital environments.
2. Combining Photogrammetry, Gaussian Splatting, and Motion Capture
-
Hybrid Productions: Assign projects where students combine motion capture (mocap) with photogrammetry-based virtual sets. They can scan environments or objects, add mocap-driven characters, and use Gaussian splatting to render everything in real-time.
-
Augmented Reality Scenes: Gaussian splatting can also be used to render real-world objects in AR, allowing students to create AR environments that blend physical and digital elements in real time.
3. Using Photogrammetry and Gaussian Splatting for Game Design
-
Open-World Game Environments: Assign projects where students use photogrammetry to capture real-world landscapes or structures and create open-world game levels. Gaussian splatting can be used to render these large, detailed environments at scale.
-
Procedural Content Creation: Introduce students to procedural techniques that can enhance photogrammetry and Gaussian splatting, allowing them to create highly dynamic, varied environments that respond to in-game interactions or real-time changes.
-
Optimizing Game Assets: Teach students how to use photogrammetry and Gaussian splatting to create game-ready assets, ensuring high detail without sacrificing performance.
Tools and Software for Photogrammetry and Gaussian Splatting
-
Photogrammetry Software:
-
RealityCapture
-
Agisoft Metashape
-
Meshroom
-
Unreal Engine's Photogrammetry Workflow
-
-
Gaussian Splatting Tools:
-
Kaolin (by NVIDIA) for implementing Gaussian splatting in real-time rendering.
-
Custom Gaussian Splatting Pipelines integrated into Unreal Engine or other 3D tools.
-
Blender (for working with point clouds and preparing assets).
-
-
Game Engines for Integration:
-
Unreal Engine: For implementing photogrammetry-based environments, Gaussian splatting, and virtual production workflows.
-
Unity: For using photogrammetry assets in game development and exploring Gaussian splatting in interactive VR/AR experiences.
-
Certification Pathways for Photogrammetry and Gaussian Splatting
To enhance career readiness, students can pursue certifications that demonstrate their skills in these technologies:
-
RealityCapture Certification (if available via RealityCapture’s platform or third-party providers).
-
Unreal Engine Certifications focused on virtual production and real-time rendering workflows, including photogrammetry integration.
-
Blender Certification with an emphasis on photogrammetry and point cloud processing.
-
Unity Certified Expert in VR/AR Development, incorporating photogrammetry and procedural rendering techniques like Gaussian splatting.
Conclusion: Enhancing the Curriculum
By incorporating photogrammetry and Gaussian splatting into your curriculum, students will:
-
Develop cutting-edge skills in real-time rendering, asset creation, and environment design.
-
Learn how to create photorealistic and efficient 3D assets for games, virtual production, VR, and AR.
-
Be prepared for future careers in game design, virtual production, film, and immersive technology industries.
-
Gain proficiency with industry-standard tools and workflows, giving them a competitive advantage in creative and technical fields.
This integration will push your program to the forefront of innovative digital production and interactive media education.
1. Addressing Educational Challenges in Game Design, Virtual Production, and Esports
The LAZR Learning Hub aims to resolve critical educational gaps in game design, virtual production, and esports, particularly in underserved and rural communities. Students often lack access to the necessary technology, mentorship, and real-world experience to prepare for careers in these industries. The LAZR Learning Hub will provide hands-on experience in Unreal Engine development, esports management, and virtual production, offering cutting-edge equipment and access to resources that are otherwise inaccessible to many.
Key challenges this project will address include:
-
Lack of access to technology: Many schools, particularly those in underserved regions, don’t have access to the hardware or software necessary for learning game development, virtual production, or esports management.
-
Limited curriculum in game design and virtual production: Schools often don’t have structured, real-world, project-based curricula that can teach both theoretical and practical aspects of game design using tools like Unreal Engine. The offered structured, project-based curriculum integrates Unreal Engine into the learning process, allowing students to use the same tools in the professional world.
-
Career pathway development: By partnering with the Ctrl V franchise and offering esports and game development projects, we will create a clear pathway for students to transition from education to industry opportunities. Along with making the many certifications, we will offer even more relevant.
2. Alignment with Epic MegaGrant Core Values
The LAZR Learning Hub aligns with the core values of the Epic MegaGrant by:
-
Expanding Unreal Engine education: This project will give students, educators, and industry professionals access to Unreal Engine through workshops, structured curriculum, and hands-on projects, supporting Epic’s goal of empowering creators.
-
Serving underserved communities: We are committed to bringing Unreal Engine and related technologies to students in underserved areas, offering access to cutting-edge learning tools and environments that they might not otherwise encounter.
-
Fostering innovation: The project promotes creativity and problem-solving through real-world projects in game design, esports, and virtual production. This innovation-focused model aligns perfectly with the values of the Epic MegaGrant.
-
Promoting creativity and innovation: Through structured project systems, students will have the opportunity to create games, virtual experiences, and esports events, fostering innovation and encouraging indie-ready developers to emerge from North Carolina.
-
Open access to resources: Our free curriculum and access to Unreal Engine training will support teachers, CTE programs, and college-level students, providing structured pathways for learning while making these resources widely available. The Hub will offer free curriculum materials, developed specifically to help educators integrate Unreal technology into their classrooms, further supporting Epic’s vision of democratizing learning.
3. Overcoming Funding Challenges
Once funded, the LAZR Learning Hub will ensure sustainability through several revenue streams:
-
Franchise partnerships: Ctrl V, EVA Esports and the Hologram Zoo will help generate a steady revenue stream through ticket sales, events, and franchised content. These projects will also draw attention to the Learning Hub and attract additional investment. We will work closely with Ctrl V to generate consistent revenue through esports events, game design showcases, and franchised experiences. These events will draw attention to the Learning Hub, generating both community engagement and financial support.
-
Professional development workshops: By offering professional development and esports training to local schools, colleges, and professional organizations, we will generate additional funding while supporting the needs of educators and industry professionals.
-
Real-world collaborations: By creating real-world, project-based learning opportunities, the Hub will be positioned to offer professional development, workshops, and esports competitions, bringing in further revenue.
-
Grant support and sponsorships: In addition to Epic’s MegaGrant, we will pursue further grants, sponsorships, and collaborations to supplement our funding and ensure longevity.
-
Revenue-sharing model: Projects developed within the Hub, such as games for Ctrl V arcades, will provide opportunities for students to generate funds and help sustain the operation of the Learning Hub.
4. Timeline of Staggered Rollout with Franchise Partners
The staggered rollout of the LAZR Learning Hub, in partnership with Ctrl V, will proceed as follows:
-
Phase 1 (Year 1-2): The Ctrl V franchise will be established to provide esports competitions, virtual reality experiences, and game development projects. During this phase, we will launch initial Unreal Engine workshops and provide access to basic curriculum for local CTE and college programs.
-
Phase 2 (Year 2-3): Expand the Learning Hub to integrate virtual production and game design courses. The Hub will offer training for educators and advanced students, with the curriculum refining and growing based on community feedback.
-
Phase 3 (Year 4): Broaden the esports and virtual production components, working with CTE programs, colleges, and local high schools to provide hands-on projects. We will also collaborate with Ctrl V on larger-scale esports competitions and game showcases.
-
Phase 4 (Year 4 and beyond): The Hub will release a full open-source curriculum, certifying students in Unreal Engine and virtual production, while supporting the growth of indie developers through game creation projects for Ctrl V arcades.
5. Qualifications to Lead This Project
I bring a wealth of experience in game design, virtual production, and educational program development, having worked with companies like Nintendo and Google, and turned around art programs at Title I schools. With expertise in Unity3D and Unreal Engine, and a history of mentoring students in both creative and technical fields, I am well-positioned to lead this project. Additionally, my successful management of a company for over a decade has given me the business acumen necessary to ensure the financial and operational success of the Learning Hub.
6. Technology Requirements
The LAZR Learning Hub will require the following technology to support its objectives:
-
High-end workstations: Capable of running Unreal Engine for game design, virtual production, and esports development, with a minimum of 20 workstations to start.
-
VR and XR hardware: For immersive esports experiences and virtual production, we will need virtual reality (VR) and mixed reality (XR) headsets to support student projects and esports events.
-
Networking and esports infrastructure: To facilitate large-scale esports competitions and real-time collaboration on game development projects, we will require servers and robust networking equipment.
-
Ctrl V-specific technology: Equipment that allows for game development testing and esports event hosting within the Ctrl V arcades.
7. Free Curriculum and Resource Support for Local Programs
The LAZR Learning Hub will offer a free, open-source curriculum designed to serve CTE and college programs in the region, providing training in Unreal Engine and access to our facility for hands-on learning. Our curriculum will cover:
-
Game design fundamentals: Introductory lessons on using Unreal Engine for game development.
-
Virtual production: Training modules focused on using Unreal Engine for filmmaking, virtual events, and architectural visualization.
-
Esports management and production: Practical lessons in organizing and managing esports events, giving students real-world experience in this growing industry.
-
Support for educators and CTE programs: By partnering with local schools and colleges, we will ensure that educators have the resources to integrate Unreal Engine into their classrooms, with SCORM-compliant content that aligns with Learning Management Systems (LMS). The Hub will also serve as a training resource, helping educators and students alike develop the skills necessary for careers in virtual production and game design.
-
Funding sustainability: Through the development of franchise partnerships like Ctrl V, we will ensure that all training and access remain free to students and educators. These partnerships will also enable us to hire paid professionals to enhance the quality of training and mentorship offered at the Learning Hub.
8. Why starting with Ctrl V matters.
We are planning a multi-purpose game design hub focused on education and production using Unreal Engine technology. It will serve as an incubator where students and community members work on real-world game development projects, blending entertainment, education, and innovative technology.
Here’s a breakdown of the key features of this hub:
Game Design and Production
-
Focus on Unreal Engine: The hub will utilize Unreal Engine, emphasizing cutting-edge VR, XR, and holographic technology for game creation. This gives students and participants access to industry-standard tools, preparing them for real-world careers.
-
Educational Collaboration: The hub will provide an environment where students can learn game design and development, focusing on teamwork, project management, and production pipelines.
-
Franchise Involvement: By leveraging franchises like Ctrl V VR arcade, Axiom’s Hologram Zoo and Eva's Esports franchise, the hub will sustain its development, ensuring high-quality content creation and engagement with a broader audience(virtualrealityfranchise)(franchisesolutions).
Incubator for Game Development
-
Real-World Skill Building: Students will work on collaborative projects that simulate real-world production environments. These projects will have the potential to reach a commercial level, with the possibility of approval for use in platforms like Ctrl V arcades(virtualrealityfranchise)(ifpg).
-
Shared Equipment: The hub will offer shared hardware and development stations for students and developers, ensuring they have access to all necessary resources to create VR and other immersive experiences(ifpg).
CTE and Community Engagement
-
Volunteers and Mentorship: The hub will involve volunteers from HBCUs, local colleges, and K-12 teachers, creating a community-driven space for learning and development.
-
Practical Integration: Students will integrate what they learn into real-world settings, producing content that solves actual problems, promotes innovation, and fosters a pool of indie-ready developers in North Carolina(
virtualrealityfranchise)(franchisegator).