Programming resources for graphics

Notes and resources for programming in 2D/3D – by Aline Normoyle

Low-level graphics APIs

TL;DR Below are resources for drawing a triangle that don’t require downloading a 5 GB game engine.

  • OpenGL – although still supported on most platforms (Linux, Windows, Mac), it is no longer actively maintained in favor of Vulcan. However, new projects, such as Zink, implement the OpenGL API on top of Vulcan.
  • webGL – An OpenGL-like Javascript API that runs in your browser. Similarly, to OpenGL, webGL is widely supported across platforms, but webGPU is recommended moving forward.
  • Vulcan – the modern, cross-platform graphics card development API
  • webGPU – Javascript API for implementing 3D graphics in your browser. It also has a native C version.
  • threejs – Javascript scene graph API for implementing 3D graphics in your browser. This API provides built-in functionality for loading models, textures, etc.
  • Cesium – Javascript scene graph API tailored for maps and 3D geospatial data in your browser.
  • p5.js/L5/Processing– high-level graphics API with 2D and 3D support. p5.js is in Javascript; L5 is in Lua; and Processing is in Java. These languages are designed for easy-of-use and fast prototyping.

A little bit of history and context

OpenGL is the original cross-platform, open standard for graphics programming. For those of us started graphics programming in the 90s and aughts, openGL 1.0 was simple, powerful, and easy to learn (and led to many people discovering a life-long love for computer graphics).

Open graphics standards, including APIs and data formats, are developed by the Khronos Group. Khronos promotes and develops interoperable standards to make graphics development easier across different operating systems and hardware. In contrast, APIs such as Direct3D (Microsoft) and Metal (Apple) are proprietary and lock you into a particular platform. If you’re interested in computer graphics, the Khronos group is an essential resource.

OpenGL, particularly in the beginning, was a fantastic API. Unfortunately, as graphics cards evolved, OpenGL grew increasingly detached from the mechanisms of the underlying hardware. This resulted in two problems: one, supporting OpenGL became increasingly difficult and complex as hardware changed; and two, OpenGL started to limit the performance capabilities of new hardware. These problems led to Vulcan as the new cross-platform graphics API. The last time I spoke with a member of the Khronos group, he told me OpenGL is dead.

Although Vulcan is fast and tailored for the modern graphics card — making it not just good for visualization but for simulation and artificial intelligence too — the API is unwieldy and complicated. With Vulcan, features that used to be implemented in the graphics driver not have to be written by you. In OpenGL 1.0, you could draw a triangle with 5 lines of code. With Vulcan, drawing the same triangle requires hundreds of lines.

Years ago, I complained about this to an engineer at AMD. His response was to use a game engine such as Unity or Unreal. It was a real “let them eat cake moment”. Luckily, there are better options in the form of lightweight libraries that hide the details of the graphics card. If you want to continue to use OpenGL, there is the Zink project which implements the OpenGL API on Vulcan. For my own work, I use my own lightweight, cross-platform libraries for simple visualizations: AGL and TinyGL. My own APIs prioritize fast prototyping (not performance) so as to make graphics programming easy. I care about easily setting different shaders, loading images and fonts, drawing meshes, and handling render targets. Your mileage may vary. Many others are also available.

Why do I maintain my own graphics library? A frustrating aspect of having so many APIs is that they are constantly shifting due to changing corporate priorities, the capriciousness of open-source projects, as well as the normal politics of community-based standards development. I’ve used dozens of graphics APIs over the past 30 years. Many are not available anymore. However, several people from Khronos have told me that they expect Vulcan and webGPU to be around for a while.

Shader APIs

Tl;DR Several shader resources

  • GLSL – A straight-forward shader language
  • Shadertoy – repository for stand-alone demos based on fragment shaders
  • Shaderpark – javascript shader platform based on Processing
  • Slang – Compiler for shaders that supports several common shader languages, including GLSL

Shaders are programs that run on the GPU, for example, to process vertices and generate pixels. Many languages also exist for writing shaders, such as GLSL, HSL, and CUDA. I prefer GLSL. NVIDIA has created SLANG as a universal compiler between shader languages.

Math and API resources

  • glm – C++ library with syntax similar to GLSL vectors and matrices. Supports 2D/3D transformations.
  • Eigen – C++ library with general matrix and vector support, including advanced linear algebra and optimization features.
  • SDL – C/C++ cross-platform media and user interface library

Video resources

Mesh model formats

  • GLTF Format optimized for the web
  • FBX Widely supported format for 3D graphics. The API is free to use.
  • OBJ Simple text-based mesh format
  • PLY Simple text-based mesh format

Modeling and Animation Tools

  • Blender
  • Pencil2D
  • Inkscape
  • Gimp