Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions docs/reference/depthestimation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,22 @@ The ml5.js DepthEstimation module offers a pretrained model for inferring depth

## Quick Start

Get up and running with the [webcam example](https://editor.p5js.org/nasif-co/sketches/Pep6DjEtD), which shows a realtime depth map estimated from the webcam video.
Get up and running with the [webcam example](https://editor.p5js.org/ml5/sketches/1AlSdDhwA), which shows a realtime depth map estimated from the webcam video.

</br>

[DEMO](iframes/depthestimation ":include :type=iframe width=100% height=550px")

## Examples
- [Webcam](https://editor.p5js.org/nasif-co/sketches/Pep6DjEtD): Show the live depth map of the video captured by the webcam.
- [Video](https://editor.p5js.org/nasif-co/sketches/vifmzXg6o): Generate the depth map of a video file as it plays.
- [Single Image](https://editor.p5js.org/nasif-co/sketches/_TcZofgrt): Depth map of an image using single-shot estimation.
- [Mask Background](https://editor.p5js.org/nasif-co/sketches/Z_1xMhUPl): Showcases how to mask out the background from the depth result.
- [Point Cloud](https://editor.p5js.org/nasif-co/sketches/VbT8hEoDz): Creates a live 3D point cloud visualization of the webcam video.
- [Mesh](https://editor.p5js.org/nasif-co/sketches/X-e1DEZr4): Creates a live 3D mesh geometry of the webcam video.
- [Webcam](https://editor.p5js.org/ml5/sketches/1AlSdDhwA): Show the live depth map of the video captured by the webcam.
- [Video](https://editor.p5js.org/ml5/sketches/YGAqvhn1o): Generate the depth map of a video file as it plays.
- [Single Image](https://editor.p5js.org/ml5/sketches/VpVUFgtOU): Depth map of an image using single frame estimation.
- [Mask Background](https://editor.p5js.org/ml5/sketches/zxi8xUuq4): Showcases how to mask out the background from the depth result.
- [Point Cloud](https://editor.p5js.org/ml5/sketches/7EOC8txJ1): Creates a live 3D point cloud visualization of the webcam video.
- [Mesh](https://editor.p5js.org/ml5/sketches/CE2f9l38k): Creates a live 3D mesh geometry of the webcam video.

## Step-by-Step Guide
### Initialization and options
### Initialization and Options
Before starting, make sure you have included the ml5 library in your `index.html` file:

```html
Expand Down Expand Up @@ -52,12 +52,12 @@ async function setup() {
}
```

### Estimating depth
As with many other ml5 models, you have two options to run depth estimation on the image, video or webcam of your choice: _Continuous Estimation_ and _Single Shot Estimation_ .
### Estimating Depth
As with many other ml5 models, you have two options to run depth estimation on the image, video or webcam of your choice: _Continuous Estimation_ and _Single Frame Estimation_ .

For any of these, make sure you first load the image, video or start the webcam capture. This is the media we will pass to the model.

#### Continuous estimation
#### Continuous Estimation
This method is used to continuously estimate depth on every frame of a video or webcam feed.
```js
// Make sure to load the model in preload or async in p5 2.0!
Expand All @@ -75,7 +75,7 @@ function gotResults(result) {
```
Using this method, the depth estimator will take care of doing estimation of a frame and waiting for it to finish before working on the next frame. Any time a depth map is ready, it will fire the callback function to provide it.

#### Single shot estimation
#### Single Frame Estimation
This method is used to estimate depth once, for a single image:
```js
// Make sure to load the image and the model in preload or asyn in p5 2.0!
Expand All @@ -90,7 +90,7 @@ function gotResults(result) {
```
Because the estimation takes time, we pass in a callback function that will fire when estimation is ready. The `estimate` method is called in setup because it **will only run once**. If calling it multiple times, it is prudent to wait for each operation to finish before starting the next one.

### Using the depth result
### Using the Depth Result
Whenever the callback function fires, we have acces to the depth result that contains all the depth information.
```js
let depthMap;
Expand Down Expand Up @@ -171,7 +171,7 @@ depthEstimator.estimateStop()
```

### depthEstimator.estimate()
This method is used for _Single Shot Estimation_: estimating depth one time on a single image or video/webcam frame.
This method is used for _Single Frame Estimation_: estimating depth one time on a single image or video/webcam frame.

```js
depthEstimator.estimate(media, callback)
Expand All @@ -197,7 +197,7 @@ These are its properties:
- Type: 2D array of floating point numbers in the 0 - 1 range.
- `mask`: The mask of the people detected in the image and for whom depth values were estimated. It can be used directly with the `mask()` function in p5.
- Type: `p5.Image` object
- `sourceFrame`: The exact frame that was analyzed to create the depth map. Because depth estimation is not immediate, the result can fall out of sync from the source video. By using this value instead of the video, the depth data is guaranteed to be in sync. See a [demo sketch](https://editor.p5js.org/nasif-co/sketches/Z_1xMhUPl) showcasing the difference.
- `sourceFrame`: The exact frame that was analyzed to create the depth map. Because depth estimation is not immediate, the result can fall out of sync from the source video. By using this value instead of the video, the depth data is guaranteed to be in sync. See a [demo sketch](https://editor.p5js.org/ml5/sketches/W8irRhKOw) showcasing the difference.
- Type: `p5.Image`
- `width`: Width of the depth map
- Type: number (integer)
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/iframes/depthestimation/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
<script>
// Navbar will be added on the top of the page.
// You can provide a link to the script file on the p5 web editor.
navbar("https://editor.p5js.org/nasif-co/sketches/Pep6DjEtD");
navbar("https://editor.p5js.org/ml5/sketches/1AlSdDhwA");
</script>

<iframe src="ready.html" name="script-iframe"></iframe>
Expand Down