JS Framebuffer

In the Drawing lesson, we learned how to send commands to the device to draw shapes and text on a display. While these drawing primitives work well for showing data that fits those requirements, they may not be sufficient for more complex rendering requirements. For example, you might want to draw complex shapes or images on the display that are not supported by the standard primitives. In such cases, you can take full control of the display and create any image you want on the host device. You can then send that frame to the target device to be immediately rendered. This approach works well if you want to render and animate complex 3D objects or even games.

Manual Framebuffer

We know that internal drawing functions render to internal memory and only a call to Show() will transfer that data (framebuffer) to the display. We will make our own framebuffer. The size of the buffer is 4 bytes per pixel, sorted as ABGR. The pixel count is 128×64 on B&W display or 160×120 on color displays.

let width = 128;
let height = 64;
let frameBuffer = new Uint8Array(160 * 120 * 4);

This is a linear array, not a 2D array like you see on the display. Still, the size is correct, 128×64 pixels. We now need a helper function to let us set pixels in the framebuffer.

function setPixel(color, x, y) {
    if (x < 0 || x > (width-1) || y < 0 || y > (height-1))
        return;

    var index = (y * width + x) * 4;
    frameBuffer[index] = (color >> 16) & 0xff;      // Red
    frameBuffer[index + 1] = (color >> 8) & 0xff;   // Green
    frameBuffer[index + 2] = color & 0xff;          // Blue
    return;
}

Let us also add a function to clear the screen. Note how we are making functions identical to the ones we use with the built in graphics.

function Clear(color){
  for(var x=0;x<width;x++){
      for(var y=0;y<height;y++){
        SetPixel(x, y, color);
  }
}

The only thing left is the Show() function, which transfers the data to the display.

async function Show()
{
    await BrainPad.Display.DrawBufferBytes(frameBuffer,1);
}

Drawing can now be done just like we did before but now the PC is doing the drawing.

Clear(0);
SetPixel(1, 10, 10);
await Show();

The results are very similar to using the internal graphics. However, using the PC’s processing power will be much faster. This code will draw random pixels around the screen. The PC can draw thousands/millions of pixels per second the update looks slow on the screen because it takes time to transfer the frambuffer from the PC to the BrainPad. When generating images, 3D, video, and more, you will be generating the entire frame on the PC and transfer to the BrainPad after.

let BrainPad = new due.DUELinkController(new SerialUSB());
await BrainPad.Connect();
let width = 128;
let height = 64;
let frameBuffer = new Uint8Array(width* height* 4);

function setPixel(color, x, y) {
    if (x < 0 || x > (width-1) || y < 0 || y > (height-1) )
        return;

    var index = (y * width + x) * 4;
    frameBuffer[index] = (color >> 16) & 0xff;      // Red
    frameBuffer[index + 1] = (color >> 8) & 0xff;   // Green
    frameBuffer[index + 2] = color & 0xff;          // Blue
}

function Clear(color){
  for(var x=0;x<width;x++){
      for(var y=0;y<height;y++){
        SetPixel(x, y, color);
  }
}

async function Show()
{
    await BrainPad.Display.DrawBufferBytes(frameBuffer, 1);
}

Clear(0);
while(true){
    var x = Math.random()*127;
    var y = Math.random()*63;
    SetPixel(200, x, y);
    await Show();
}

Do you know how to draw circles? Lines? Use the SetPixel() method to add them.

---

Using PC’s Graphics

The previous work got the bases ready for the user to implement their own drawing functions. We however can use the graphics implementation found on the system being used. This highly depends on what system and what libraries are being used. In this demo, we are using the canvas library. This example uses the system’s font. This allows users to draw international fonts on the screen.

// Install the canvas library
// npm install canvas

import {SerialUSB} from './serialusb.js';
import * as due from './duelink.js';
import {createCanvas} from 'canvas';

let BrainPad = new due.DUELinkController(new SerialUSB());
await BrainPad.Connect();

const canvas = createCanvas(BrainPad.Display.Width, BrainPad.Display.Height)
const ctx = canvas.getContext('2d')

let x=10;
let y=15;
let dx=5;
let dy=4;

for(;;) {
    // Clear canvas
    ctx.fillStyle = 'black';
    ctx.fillRect(0, 0, 128, 64);

    // Write "Awesome!"
    ctx.fillStyle = 'white';
    ctx.font = '12px Arial'
    ctx.fillText('Awesome!', x, y)

    let pixelData = ctx.getImageData(0,0,128,64).data;
    await BrainPad.Display.DrawBuffer(pixelData);

    x+=dx;
    y+=dy;
    if (x < 0 || x > 63) dx = -dx;
    if (y < 0 || y > 63) dy = -dy;
}

We can load images on supported color displays as well. The image can be transferred 4BPP, 8BPP, or 16BPP.

import {SerialUSB} from './serialusb.js';
import * as due from './duelink.js';
import { createCanvas, loadImage } from 'canvas';

let BrainPad = new due.DUELinkController(new SerialUSB());
await BrainPad.Connect();

const canvas = createCanvas(BrainPad.Display.Width, BrainPad.Display.Height);
const ctx = canvas.getContext('2d');
const img = await loadImage("Flower.png");

ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
let pixels = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
await BrainPad.Display.PaletteFromBuffer(pixels); //only needed for 4BPP
await BrainPad.Display.DrawBuffer(pixels, 4); //4BPP, try 16 as well

//close serial com
await BrainPad.Disconnect()

---

What’s Next?

Do you have experience with drawing on computers? Build 3D drawing on the screen.

BrainStorm

Graphics have always been a challenge to computers’ processing power. We have gone from Pong games in the 70s to having real-life-looking graphics in a few decades. The BrainPad takes you all the way back to the 70s and brings you on a journey to see how pixels evolved into shapes, into fake-3D to real-3D and beyond!