Hi!

3D Graphics on Android:
Lessons from Google Body

Nico Weber
May 11, 2011

Demo
bodybrowser.googlelabs.com

Google Body

Demo
Google Body for Android

I’m mostly clueless.

Ideally, so are you.

Completely clueless? Read http://code.google.com/p/gdc2011-android-opengl/

Google Body for Android

Agenda

  1. OpenGL ES 2.0
  2. GPUs
  3. Textures
  4. Vertex Buffer Objects
  5. Reading data
  6. Performance tweaking

Part 0: OpenGL ES 2.0

OpenGL

glBegin(GL_TRIANGLES);
  glVertex3f(0, 0, 0);
  glVertex3f(1, 0, 0);
  glVertex3f(0, 1, 0);
glEnd();

The John Carmack of 3d apis.

Roughly the same age, too.

OpenGL ES

(WebGL)

GLSurfaceView

GLSurfaceView

  // In your Activity
  @Override
  public void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      mView = new GLSurfaceView(this);
      mView.setEGLContextClientVersion(2);
      mView.setRenderer(new MyFineRenderer());
      setContentView(mView);
  }

  @Override
  protected void onPause() { super.onPause(); mView.onPause(); }

  @Override
  protected void onResume() { super.onResume(); mView.onResume(); }

GLSurfaceView

<!-- In AndroidManifest.xml -->
<uses-feature
  android:glEsVersion="0x00020000"
  android:required="true" />

Renderer

import static android.opengl.GLES20.*;
public class MyFineRenderer implements GLSurfaceView.Renderer {
  @Override
  public void onSurfaceCreated(GL10 unused, EGLConfig config) {
  }

  @Override
  public void onDrawFrame(GL10 unused) {
      // Do actual drawing, calculate fps, etc
  }

  @Override
  public void onSurfaceChanged(GL10 unused, int w, int h) {}
}

Communicate UI thread → renderer thread

Use GLSurfaceView.queueEvent:

// In Activity, on UI thread.
void onClick(DialogInterface dialog, int item) {
  mGLView.queueEvent(new Runnable() {
      public void run() {
        // Runs on renderer thread.
        mGLView.getRenderer().setDrawsSomeFunkySelection();
      }});
}

Communicate renderer thread → UI thread

Use Activity.runOnUiThread:

// on renderer thread
void ohHeyDoneUploadingData() {
  mActicity.runOnUiThread(new Runnable() {
      public void run() {
        // Runs on UI thread.
        mActivity.setSearchBoxEnabledAndWhatNot();
      }});
}

Advice: Use GLSurfaceView

Beware: GLSurfaceView loses its GL context often

Part 1: Rough mental model of GPUs

Part 2: Textures

OpenGL 101: Don't upload textures every frame

void onDrawFrame(GL10 unused) {
  glTexImage2D(..., GL_RGBA, ..., texData);  // DON'T
  drawModel()
}
// Better:
void onSurfaceCreated(GL10 unused, EGLConfig config) {
  int[] texture = { 0 };
  glGenTextures(1, texture, 0);
  glBindTexture(GL_TEXTURE_2D, texture[0]);
  glTexImage2D(..., GL_RGBA, ..., texData);
}
void onDrawFrame(GL10 unused) {
  glBindTexture(GL_TEXTURE_2D, texture[0]);
  drawModel();
}

Advice: Use ETC for RGB texture compression

Save 75% memory, or have twice the texture resolution for the same memory.

android-sdk/tools/etc1tool converts pngs to pkm

<supports-gl-texture
    android:name="GL_OES_compressed_ETC1_RGB8_texture" />

ETC loading code

// ETC1Util requires Android 2.2+
ETC1Texture t = ETC1Util.createTexture(
  getResources().openRawResource(
      R.raw.mytexture));  // I/O thread

ETC1Util.loadTexture(..., t);  // GL thread

Beware: Only ETC textures with w, h being a multiple of 4 look right on PowerVR

Part 3: VBOs

OpenGL ES 101: Don't upload vertex data every frame

void onDrawFrame(GL10 unused) {  // NO NO NO
  glVertexAttribPointer(..., GL_FLOAT, 3, ..., attribData);
  glDrawElements(GL_TRIANGLES, ..., indexData);
}

OpenGL ES 101: Don't upload vertex data every frame

// Better (just attribs shown, indices part looks similar):
void onSurfaceCreated(GL10 unused, EGLConfig config) {
  attribVBO = glGenBuffers(1);
  glBindBuffer(GL_ARRAY_BUFFER, attribVBO);
  glBufferData(..., attribData ...);
  glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexVBO);
  glBufferData(..., indexData ...);
}
void onDrawFrame(GL10 unused) {
  glBindBuffer(GL_ARRAY_BUFFER, attribVBO);
  glVertexAttribPointer(..., GL_FLOAT, 3, ..., 0); // !
  glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexVBO);
  glDrawElements(GL_TRIANGLES, ..., 0);  // !
}

Beware: OpenGL ES 2.0 Java bindings incomplete on Android 2.2

Annoying, but easy to fix.

…if you’re familiar with the NDK

// src/com/example/yourapp/GLES20Fix.java
public class GLES20Fix {
  native public static void glDrawElements(
      int mode, int count, int type, int offset);

  native public static void glVertexAttribPointer(
      int index, int size, int type, boolean normalized,
      int stride, int offset);

  private GLES20Fix() {}
  static {
      System.loadLibrary("GLES20Fix");
  }
}
// jni/GLES20Fix.c
#include <jni.h>
#include <GLES2/gl2.h>

// |extern "C"| if you're using C++
void Java_com_example_io_GLES20Fix_glDrawElements(
        JNIEnv *env, jclass c, jint mode,
        jint count, jint type, jint offset) {
    glDrawElements(mode, count, type, (void*) offset);
}

// Same for other method
# jni/Android.mk
LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE    := GLES20Fix
LOCAL_SRC_FILES := GLES20Fix.c
LOCAL_LDLIBS    := -lGLESv2

include $(BUILD_SHARED_LIBRARY)
cd jni
ndk-build
# Then do a clean build in Eclipse to make it pick up the new .so

Part 4: Filling direct ByteBuffers

Direct ByteBuffers

Wrong

// DO NOT DO THIS.
InputStream is = getResources().openRawResource(R.raw.myresource);
    // Or |getAssets().open("my.resource")|
int size = is.available(), v;
ByteBuffer b = ByteBuffer.allocateDirect(size);
while ((v = is.read()) != -1)
    b.put(v);  // OMG ELMENT-WISE ACCESS, SLOW
// A buffer has an internal "current index"
b.rewind();

Better

// This is ok.

// In a real app, use a block size and a loop.
byte[] buff = new byte[size];
is.read(buff);
b.put(buff);

Faster still

Faster still

AssetFileDescriptor ad = getAssets().openFd("myfile.jet");
    // or |getResources().openRawResourceFd(id)|
FileInputStream fis = ad.createInputStream();
FileChannel fc = fis.getChannel();
// Always direct.
MappedByteBuffer b = fc.map(
    MapMode.READ_ONLY,
    ad.getStartOffset(),
    ad.getLength());
long size = ad.getLength();
// upload b here

Beware: ByteBuffer.allocateDirect() overallocates

Tears. Not much you can do about it. Keep your buffers small.

Or use NDK & own bytebuffer-less bindings for upload.

Beware: Compressed files can be max 1MB uncompressed on ≤ Android 2.2

Tears. Split files.

Or roll your own compression.

Part 5: Performance

Measure.

Demo

public void onSurfaceCreated(GL10 unused, EGLConfig config) {
    //GLES20Fix.setPreserve();
}
public void onDrawFrame(GL10 unused) {
    for (int i = 0; i < 7; ++i)
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
}

Beware: Double-buffering skews measurement

#include <egl/egl.h>  // -lEGL in Android.mk

void Java_com_example_io2011_GLES20Fix_setPreserve() {
  #define EGL_SWAP_BEHAVIOR               0x3093
  #define EGL_BUFFER_PRESERVED            0x3094

  eglSurfaceAttrib(  // Possibly during dev only, for measuring.
      eglGetCurrentDisplay(), eglGetCurrentSurface(EGL_DRAW),
      EGL_SWAP_BEHAVIOR, EGL_BUFFER_PRESERVED);
}

Recap

Fragment processor bound

How to know: Reducing texture sizes doesn't help, but glViewport(0, 0, 1, 1) does

Texture fetch bound

How to know: Using much smaller textures and changing nothing else is way faster.

Vertex processor bound

How to know: Lower-res model that covers about the same number of pixels is way faster / small viewport isn’t

CPU bound

How to know: traceview and top shows your code as expensive.

Thank you!

#io2011 #android