Google Cardboard on Processing for Android

Today I worked on making stereoscopic images with Processing for the Android phone. The code below displays an image in two viewports – one for the left eye and one for the right eye. The result is that the image looks 3D when viewed from a Google Cardboard device. Accelerometer and gyroscope data are used to move the 3D image as the head is moved around. The only bug is that of Processing for Android in that Landscape mode makes the program crash if you do not start it in this mode. I am using Processing 2.0.3 and Android 4.3, so this problem may have been addressed in current versions. (Although I did see it was still an open issue in Processing-Bugs discussion on Github). The texture image is a 100 x 100 pixel image of a favorite cartoon character. You can use whatever you want – just store the image in the data folder. If you have suggestions for improvement, I’d love to hear it!

Update (3/1/15): I was able to fix the orientation problem by removing the orientation(LANDSCAPE) line and adding android:screenOrientation=”landscape” in the XML file as described here.

pboard_img

Here’s the code:

[java]
//Scott Little 2015, GPLv3
//pBoard is Processing for Cardboard

import android.os.Bundle; //for preventing sleep
import android.view.WindowManager;
import ketai.sensors.*; //ketai library for sensors
KetaiSensor sensor;

float ax,ay,az,mx,my,mz; //sensor variables
float eyex = 50; //camera variables
float eyey = 50;
float eyez = 0;
float panx = 0;
float pany = 0;
PGraphics lv; //left viewport
PGraphics rv; //right viewport
PShape s; //the object to be displayed

//********************************************************************
// The following code is required to prevent sleep.
//********************************************************************
void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// fix so screen doesn’t go to sleep when app is active
getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
}
//********************************************************************

void setup() {
sensor = new KetaiSensor(this);
sensor.start();

size(displayWidth,displayHeight,P3D); //used to set P3D renderer
orientation(LANDSCAPE); //causes crashing if not started in this orientation

lv = createGraphics(displayWidth/2,displayHeight,P3D); //size of left viewport
rv = createGraphics(displayWidth/2,displayHeight,P3D);

PImage img = loadImage(“jake.jpg”);
s = createShape();
TexturedCube(img, s, 50, 50);
}

void draw(){
//draw something fancy on every viewports
panx = panx-mx*10;
pany = 0;
eyex = 0;
eyey = -20*az;

ViewPort(lv, eyex, eyey, panx, pany, -15);
ViewPort(rv, eyex, eyey, panx, pany, 15);

//add the two viewports to your main panel
image(lv, 0, 0);
image(rv, displayWidth/2, 0);
}

void onAccelerometerEvent(float x, float y, float z){
ax = x;
ay = y;
az = z;
}

void onGyroscopeEvent(float x, float y, float z){
mx = x;
my = y;
mz = z;
}

void ViewPort(PGraphics v, float x, float y, float px, float py, int eyeoff){
v.beginDraw();
v.background(102);
v.lights();
v.pushMatrix();
v.camera(x+eyeoff, y, 300, px, py, 0, 0.0, 1.0, 0.0);
v.noStroke();
//v.box(100);
v.shape(s);
v.popMatrix();
v.endDraw();
}

void TexturedCube(PImage tex, PShape s, int a, int b) {
s.beginShape(QUADS);
s.texture(tex);

// +Z “front” face
s.vertex(-a, -a, a, 0, b);
s.vertex( a, -a, a, b, b);
s.vertex( a, a, a, b, 0);
s.vertex(-a, a, a, 0, 0);

// -Z “back” face
s.vertex( a, -a, -a, 0, 0);
s.vertex(-a, -a, -a, b, 0);
s.vertex(-a, a, -a, b, b);
s.vertex( a, a, -a, 0, b);

// +Y “bottom” face
s.vertex(-a, a, a, 0, 0);
s.vertex( a, a, a, b, 0);
s.vertex( a, a, -a, b, b);
s.vertex(-a, a, -a, 0, b);

// -Y “top” face
s.vertex(-a, -a, -a, 0, 0);
s.vertex( a, -a, -a, b, 0);
s.vertex( a, -a, a, b, b);
s.vertex(-a, -a, a, 0, b);

// +X “right” face
s.vertex( a, -a, a, 0, 0);
s.vertex( a, -a, -a, b, 0);
s.vertex( a, a, -a, b, b);
s.vertex( a, a, a, 0, b);

// -X “left” face
s.vertex(-a, -a, -a, 0, 0);
s.vertex(-a, -a, a, b, 0);
s.vertex(-a, a, a, b, b);
s.vertex(-a, a, -a, 0, b);

s.endShape();
}
[/java]

3 thoughts on “Google Cardboard on Processing for Android

  1. Hi Scott,
    I am trying to run this code on my Nexus 5 with Android 5.1 and processing 2.1 and I am getting a nasty openGL error. I decided to flash an older android version 4.4.4 and building from processing 2.0.3 and I am still getting a GL error (a different one). Could you tell me what phone you were running it on and what was the error exactly with the landscape mode off? The landscape mode fix didn’t work for me.
    Thanks,
    Anton

  2. I am running it on an S3 with Android 4.4.4 (Cyanogenmod 11).

    I didn’t realize the link above was broken (“…as describe here”). Try the XML file here: https://github.com/scottlittle/pBoard. If you copy from github, make sure you get everything. So, it may just be better to download it.

    The error I would get when the orientation didn’t work was just that the application had crashed.

    I initially had an OpenGL error too. I can’t remember what I did to resolve it. I remember playing around with different renderer modes (i.e., P3D or OPENGL), but given that your setup is very close to mine, I’m not sure what it could be.

    Good luck and let me know if you find out!

  3. Hello Scott,

    I tried to adapt the initial code to my processing project but it triggered the following errors:

    FATAL EXCEPTION: GLThread 1873
    Process: processing.test.androidproj, PID: 14032
    java.lang.NullPointerException
    at processing.opengl.PGLES.texParameterf(Unknown Source)
    at processing.opengl.Texture.allocate(Unknown Source)
    at processing.opengl.Texture.init(Unknown Source)
    at processing.opengl.Texture.(Unknown Source)
    at processing.opengl.FontTexture.addTexture(Unknown Source)
    at processing.opengl.FontTexture.initTexture(Unknown Source)
    at processing.opengl.FontTexture.(Unknown Source)
    at processing.opengl.PGraphicsOpenGL.textLineImpl(Unknown Source)
    at processing.core.PGraphics.textLineAlignImpl(Unknown Source)
    at processing.core.PGraphics.text(Unknown Source)
    at processing.core.PGraphics.text(Unknown Source)
    at processing.test.androidproj.androidproj.draw(androidproj.java:149)
    at processing.core.PApplet.handleDraw(Unknown Source)
    at processing.opengl.PGLES$AndroidRenderer.onDrawFrame(Unknown Source)
    at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1532)
    at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1249)

    Do you know why this happened?

    Thank you for the help

Leave a Reply

Your email address will not be published. Required fields are marked *