ARCore Sceneform SDK: Getting Started

Displaying XML in Real Life

What would a game be without a scoreboard? You’re going to create one using an XML-based model, just like any regular Android view.

In the starter project, there’s already a class called ScoreboardView, which inflates scoreboard_view.xml, so you’re simply going to create the 3D model from it. Add a new property scoreboardRenderable to MainActivity and update initResources() to set a value for the existing scoreboard property and then build scoreboardRenderable using scoreboard:

private var scoreboardRenderable: ViewRenderable? = null

private fun initResources() {
  ...
  scoreboard = ScoreboardView(this)

  scoreboard.onStartTapped = {
    // Reset counters
    scoreboard.life = START_LIVES
    scoreboard.score = 0
  }

  // create a scoreboard renderable (asynchronous operation,
  // result is delivered to `thenAccept` method)
  ViewRenderable.builder()
      .setView(this, scoreboard)
      .build()
      .thenAccept {
        it.isShadowReceiver = true
        scoreboardRenderable = it
      }
      .exceptionally { it.toast(this) }
}

Similarly to what you’ve done for the previous renderable, you have provided a source for the renderable but are using setView this time instead of setSource.

You’re also telling the renderable that it has the capability of receiving shadows casted by other objects with isShadowReceiver = true.

If you check what onStartTapped does in ScoreboardView, you see it’s just using an onClickListener to intercept a click event, since this 3D object works exactly like a regular view.

Adding Another Light

Afraid of the dark? Sceneform can help you! Instantiating a light is so light — no pun intended — that this operation is done synchronously by default. Add a new property failLight to MainActivity and set it up at the bottom of initResources():

private var failLight: Light? = null

private fun initResources() {
  ...
  // Creating a light is NOT asynchronous
  failLight = Light.builder(Light.Type.POINT)
      .setColor(Color(android.graphics.Color.RED))
      .setShadowCastingEnabled(true)
      .setIntensity(0F)
      .build()
}

As you can easily see, it’s possible to build different types of light with different colors, intensity and other parameters. You can read more about this here.

In the above code snippet, you just created a Light object from its builder and set the following characteristics for it:

1. Light color to Red.
2. Light intensity to zero.
3. Shadow-casting property of light to true.

This light will blink when the player fails to hit a droid, so the intensity is equal to zero for now.

Interacting With the AR World

Every object that is displayed on the plane is attached to a Node. When an OnTapArPlaneListener is applied on the plane and a user taps on it, a HitResult object is created. Through this object, you can create an AnchorNode. On an anchor node, you can then attach your own node hosting your renderable. In this way, you can use a custom node that can be moved, rotated or scaled.

For WhacARDroid, you’ll need a node capable of moving up and down, so you’re going to create the class TranslatableNode. You’ll add the animation code later; for now, a helper method for adding some offset for translation is enough.

Right-click on the main app package and choose New ▸ Kotlin File/Class and then create the new class:

class TranslatableNode : Node() {

  fun addOffset(x: Float = 0F, y: Float = 0F, z: Float = 0F) {
    val posX = localPosition.x + x
    val posY = localPosition.y + y
    val posZ = localPosition.z + z

    localPosition = Vector3(posX, posY, posZ)
  }
}

Make sure that any imports you pull in for the new class are from com.google.ar.sceneform sub-packages.

Touch Listener on a 3D Object

Like with other listeners on Android, an event is propagated until a listener consumes it. Sceneform propagates a touch event through every object capable of handling it, until it reaches the scene. In the order that follows, the event is sent to:

Knowing the theory, it’s now time to plan the game logic:

Go ahead and add two new properties to MainActivity, one to represent the grid of droids and the other to indicate whether the game board is initialized, and then add the rest of the following code to the end of onCreate(), after the call to initResources:

• scene.setOnPeekTouchListener(): This listener cannot consume the event.
• onTouchEvent() of the first node intercepted on the plane. If it does not have a listener, or if its listener return false, the event is not consumed.
• onTouchEvent() of every parent node: Everything is handled like the previous node.
• scene.onTouchListener(): This handles the touch action when the last parent is reached, but it doesn’t consume the event.

Knowing the theory, it’s now time to plan the game logic:

• When the player taps the plane for the first time, Sceneform will instantiate the whole game, so you’ll set an onTouchListener on the plane.
• When the player taps on the Start button, the game will begin; the click listener is handled by the onClickListener of ScoreBoardView.
• If the player hits a droid, he or she will gain 100 points, so you’ll intercept the onTouchEvent on the droid.
• If the player misses the droid, he or she will lose a life and 50 points; you’ll need to detect a tap on the plane so you can reuse the existing listener.

Go ahead and add two new properties to MainActivity, one to represent the grid of droids and the other to indicate whether the game board is initialized, and then add the rest of the following code to the end of onCreate(), after the call to initResources:

• When the player taps the plane for the first time, Sceneform will instantiate the whole game, so you’ll set an onTouchListener on the plane.
• When the player taps on the Start button, the game will begin; the click listener is handled by the onClickListener of ScoreBoardView.
• If the player hits a droid, he or she will gain 100 points, so you’ll intercept the onTouchEvent on the droid.
• If the player misses the droid, he or she will lose a life and 50 points; you’ll need to detect a tap on the plane so you can reuse the existing listener.

private var grid = Array(ROW_NUM) { arrayOfNulls<TranslatableNode>(COL_NUM) }
private var initialized = false

override fun onCreate(savedInstanceState: Bundle?) {
  ...
  arFragment.setOnTapArPlaneListener { hitResult: HitResult, plane: Plane, _: MotionEvent ->
    if (initialized) {
      // 1
      // Already initialized!
      // When the game is initialized and user touches without
      // hitting a droid, remove 50 points
      failHit()
      return@setOnTapArPlaneListener
    }

    if (plane.type != Plane.Type.HORIZONTAL_UPWARD_FACING) {
      // 2
      // Only HORIZONTAL_UPWARD_FACING planes are good to play the game
      // Notify the user and return
      "Find an HORIZONTAL and UPWARD FACING plane!".toast(this)
      return@setOnTapArPlaneListener
    }

    if(droidRenderable == null || scoreboardRenderable == null || failLight == null){
      // 3
      // Every renderable object must be initialized
      // On a real world/complex application
      // it can be useful to add a visual loading
      return@setOnTapArPlaneListener
    }

    val spacing = 0.3F

    val anchorNode = AnchorNode(hitResult.createAnchor())

    anchorNode.setParent(arFragment.arSceneView.scene)

    // 4
    // Add N droid to the plane (N = COL x ROW)
    grid.matrixIndices { col, row ->
      val renderableModel = droidRenderable?.makeCopy() ?: return@matrixIndices
      TranslatableNode().apply {
        setParent(anchorNode)
        renderable = renderableModel
        addOffset(x = row * spacing, z = col * spacing)
        grid[col][row] = this
        this.setOnTapListener { _, _ ->
          // TODO: You hit a droid!
        }
      }
    }

    // 5
    // Add the scoreboard view to the plane
    val renderableView = scoreboardRenderable ?: return@setOnTapArPlaneListener
    TranslatableNode()
            .also {
              it.setParent(anchorNode)
              it.renderable = renderableView
              it.addOffset(x = spacing, y = .6F)
            }

    // 6
    // Add a light
    Node().apply {
      setParent(anchorNode)
      light = failLight
      localPosition = Vector3(.3F, .3F, .3F)
    }

    // 7
    initialized = true
  }
}

In the above listener, you:

1. Handle a failed hit on a droid, and return.
2. Alert the user if they’ve picked a bad plane for the game, and return.
3. Return if not all renderable objects have been initialized.
4. Set up droids on the plane.
5. Add the scoreboard view to the plane.
6. Add a light to the game.
7. Set initialized to true.

With that long snippet added, you can now try to run the app. If everything is OK, you should be able to spawn the game by touching a plane.

Nothing will happen right now, however; you’ll need to add some logic.