WebXR: A Contributor’s Story with Zhixiang Teoh

The “Contributor’s Story” series is intended to provide a face and voice to our major contributors and community members, an overview of the projects they are working on, and the successes and challenges contributors face when developing.

In this blog post, we will be talking to Zhixiang Teoh, a WebXR contributor working with 3D media layers in a virtual environment through the Major League Hacking (MLH) Fellowship.

“For the Fellowship, I am coding something fun, new, and different, and that I know is for the open source community.”

Tell us a little bit about yourself and your current experience in the MLH Fellowship.

I'm a sophomore studying Computer Science and Math at the University of Michigan. I'm currently an open source Fellow contributing to the Facebook/WebXR project, working with 3D media layers in a virtual environment. I code in Java, C++, Python, JavaScript, and dabble in other paradigms like functional programming in Haskell and logic programming in Prolog (my show & tell was on Prolog). I have recently been working most heavily on frontend development, in React and three.js, but am currently focusing efforts in diving deeper into DevOps (recreational interest) and programming languages (research interest).

I first encountered the MLH Fellowship in July of 2020, but didn't initially think much about applying, because of the pandemic and the fact that I was dealing with other decisions to make like completing a semester at a local university in Singapore (instead of coming back to the US where I'm currently at). Halfway through my semester, I was applying some newly learned JavaScript and web development skills in personal projects, and I stumbled across multiple Medium and Dev.to posts about the Fellowship and the wonderful experiences some of the fellows had. On Reddit, people were also talking about how it's a unique experience and definitely worth applying to. Seeing these rave reviews made me decide to apply, and I just gave it a shot, despite initially being worried that I did not have much "hacking" or "open source" experience, even though I was applying to the Open Source track. I have not regretted my decision! The Fellowship is certainly one of the highlights thus far of my college experience.

Where did you first learn about open source? How did you get started using/contributing?

Certainly via GitHub. I'd occasionally stumble upon ideas for personal projects (in the shower or whenever I'm not glued to my phone and have time for my mind to really wander), and then I'd google the idea, and more often than not, "it's already been done". And on GitHub. I'd star the repository, sometimes fork it, play around with it while procrastinating on school work, and from one repository jump to another, thereby discovering other open source projects.

I hadn't really contributed to big open source projects prior to the Fellowship (and even during the Fellowship not exactly to a large repository), but I had left some comments on PRs and Issues in small repositories. While I didn't get the traditional experience of contributing directly to a large codebase, I certainly learned a lot in terms of code collaboration, both with peers and mentors, through the Fellowship, which helped me grow as a developer and hacker.

Describe the project you are currently working on.

The project I'm currently working on, with my podmate Soham, is creating WebXR samples that interface with the new WebXR media layers specification to create and interact with independent video layers in a virtual environment. These layers are essentially standalone videos, of different 3D characteristics--like "equirectangular" which is a 360 spherical immersive video, "quad" which is like a flat TV screen, and "cylinder" which is basically like a curved TV screen. The traditional way to create the same effect of having different media layers is taxing on the renderer and CPU. WebXR media layers provide a way to reduce the dependence on CPU and leverage the GPU to create fluid equirectangular background media layers and any number of quad and cylinder layers on top of it.

Creating these video layers is straightforward, but much of the work has been leveraging the three.js 3D rendering library to create an interface for controller and hand interactions with these layers, so that a user has access to all the regular playback controls of a video, and more--scrubbing, moving these videos anywhere in 3D space and resizing.

How did you initially go about tackling the issue?

We were initially tasked with working on existing code contributed by someone previously. However, we quickly decided to create entirely new samples ourselves to make use of the three.js 3D rendering library in our strive towards a cleaner and better solution that hopefully can eventually be added as a sample in the three.js library.

What roadblocks or problems have you faced thus far in your contribution?

WebXR is a particularly niche area, even within 3D web development. The WebXR layers specification, especially, is rather new. Questions regarding WebXR layers and WebXR in general cannot be as easily answered as a simple search on StackOverflow. This has certainly pushed my teammate and me out of our comfort zones, and taught us to confide in each other and tackle problems together. In particular, direct interaction with WebXR layers is not currently possible, so we have had to use workarounds, by way of 3D objects, to produce the desired effect. This has certainly called for me to think more creatively and not as mechanically as in some other previous personal projects I've done.

What is the current status of development?

We are about 70% through the goals set out for us, but are proud of what we have accomplished so far. One focus my teammate and I have placed in working on the project is code cleanliness and maintainability. Refactoring has been a big part of our time commitment to the project, but not for naught. We very much intend for our contribution to be able to be used by both future users interested in creating WebXR video samples, and potentially other 3D rendering libraries.

What have you learned about the project, development, or open source thus far?

WebXR is much less scary than I initially thought it would be, and working on this project has definitely given me confidence to tackle new challenges in my future endeavors. Again, I have not had the opportunity to contribute to an open source repository where my PRs would be slammed and attacked left, right, and center by ten reviewers, but this project has given me the confidence to approach other open source projects. Apart from the technical skills I've picked up from my awesome teammate, I have learned to be more responsible for my work and also take responsibility and pride in not only the code I write but the code I review and merge. After all, as a contributor, I should be aiming to "own" any and all aspects of the codebase, and take pride in our collective work.

What advice would you give future contributors to the open source project?

For 3D web development and WebXR, the three.js library is your best friend. Three.js is an awesome library with huge amounts of support. Also, very importantly, no solution is considered "bad" in this project! If it solves the problem well, it will always be welcome as a solution to employ, until better solutions are found. WebXR is extremely exciting! It can be really daunting at first, with the boilerplate code and syntax and all, but remember: three.js examples are your best friends.

We’d like to thank Zhixiang for their continuous contributions to the Facebook Open Source ecosystem. You can follow Zhixiang’s work through GitHub.

If you’d like to learn more about Facebook Open Source, follow us on Twitter, Facebook, and YouTube for relevant updates, and check out the WebXR website for how to get started.