Integrate react-native-skia in a ECS / Gameloop

[bastischulz]

Hello everyone,
I'm in the process of building a simple (multiplayer) game. (Agar.io optics) I came across the following problem:

I have many game objects that are added and removed - so a dynamic list of a large number of objects that are on the screen at the same time.

My game logic runs in a game loop at a fixed rate because the logic has to run later on the client and on the server.

Now my question: How (via which api) can I best bring all my game objects and my hud data efficiently to the skia canvas in my RenderSystem - i.e. preferably without setState from React, but on the ui thread?



The Picture, Atlas and offscreen rendering all don't seem to be 100% the right approach, or am I looking at it wrong?

I have react-native-reanimated installed already.



Thank you in advance!

[LoserAntbear]

Hey, were you able to find an answer for that?

[bastischulz]

Not yet. But I did not try. tbh.

[Hau-Hau]

I was building a very similar worklet based ecs system but for rendering target I've chose normal non skia react-native views.
The key to updating the view without state updates is to have a listener/useDerivedValue in each of view renderers. You will get best results and flexibility if your ECS system code will be 100% worklet based.
Your views should then be reactive to the state updates.
You can create a monolithic state and filter dirty entities or create shared value per entity.
I've gone with a monolithic state as overall performance is good and ecs is less complicated.
Also, I really recommend to implement object pool pattern that would manage your view renderers. With object pool pattern you will omit high cost of spawning/despawning new entities on screen.

update:
Out of curiosity I have implemented two additional rendering pipelines that targets skia's declarative and imperative api.

For my tests I'm using Sony Xperia 10 IV with Snapdragon 695 which is considered rather budget CPU.

From my observations declarative SKIA target has more stable FPS than react-native+reanimated, but it doesn't really matter as both rendering pipelines have exactly the same bottleneck which is spawning/despawning views on screen.

Declarative SKIA have much more stable FPS during simultaneous animation, but the process of spawning views above 128 elements on screen can drop FPS to 8 frame rates. It stabilises later, but such frame drops are unacceptable and makes such rendering target useless for any game that operates on many quickly spawned/despawn entities.
Even a declarative object pool pattern is not really helpful above 128 elements.

On another hand, I have implemented a rendering pipeline that targets skia's imperative API.
This pipeline is outstanding in terms of performance, amount items on screen and FPS stability.
It smoothly scaled to 1024 simultaneously animated rects without any problem and kept stable 30 FPS even then. I saw maybe some frame drops during expanding object pool capacity, but it is less or more expected.
Also worth mentioning is fact that this imperative API renders every entity on every frame, I do not even implement any mechanism to redraw only dirty entities yet.

TLDR; ensure your ECS system is working on worklets and use skia's imperative API, this should give you quite good results.

[bastischulz]

Do you have a link or a code example to the imperative api?

[Hau-Hau]

Sure, I will leave here some samples that should give you idea how to structure your code.

Below you will find simple renderer example. canvas: SkCanvas gives you full control on rendering.

const renderFrame = (
  canvas: SkCanvas,
  renderers: { [key: string]: ImperativeRenderer<any, any> },
  world: World<any, any>,
) => {
  "worklet";
  
  canvas.clear(Skia.Color("transparent"));
  
  const viewTypeComponents = world.currentComponents.viewType;
  if (!viewTypeComponents) return;

  for (const entity of world.entities) {
    const viewType = viewTypeComponents[entity];
    if (!viewType) continue;
    
    const renderer = renderers[viewType];
    if (!renderer) continue;

    const entityComponents = {};
    for (const archetype of renderer.archetypes) {
      (entityComponents as any)[archetype] = world.currentComponents[archetype][entity];
    }

    canvas.save();
    renderer.render(canvas, entity, entityComponents, world);
    canvas.restore();
  }
};  

export const SkiaCanvas = (props) => {
  const { world } = useECS(props);
  const frame = useSharedValue<SkPicture>(createPicture(() => {}))
  const bounds = useSharedValue(Skia.XYWHRect(0, 0, Dimensions.get("window").width, Dimensions.get("window").height));

  useDerivedValue(() => {
    "worklet";
    const recorder = Skia.PictureRecorder();
    const canvas = recorder.beginRecording(bounds.value);
    renderFrame(canvas, props.renderers, world.value);
    frame.value = recorder.finishRecordingAsPicture();
  }, [world]);

  return (
    <Canvas style={{ flex: 1 }}>
      <Picture picture={frame} />
    </Canvas>
  );
}

And below you will find example of simple view renderer:

const createRectangleRenderer = <TComponents extends Components>() => {
  "worklet";
  
  const paint = Skia.Paint();
  paint.setStyle(PaintStyle.Fill);
  return {
    archetypes: ["position", "size", "color"],
    render: (canvas: SkCanvas, entity: Entity, components: Partial<{ [K in keyof TComponents]: TComponents[K][Entity] }>, world: World<TestComponents, TestResources>) => {
      "worklet";
      
      const position = components.position;
      const size = components.size;
      const color = components.color;
      
      if (!position || !size || !color) return;
      
      paint.setColor(Skia.Color(color));
      
      const x = position.x - size.width / 2;
      const y = position.y - size.height / 2;
      
      const rect = Skia.XYWHRect(x, y, size.width, size.height);
      canvas.drawRect(rect, paint);
    },
  };
};

If you will also ensure that your ECS system is not going back and forth between UI and JS thread and will keep everything in worklet space you can expect good results.