3D Highlighter - 42Papers

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

3D Highlighter

3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions

We present 3d highlighter, a technique for localizing semantic regions on a mesh using text as input.

Our method contextualizes the text description using a neural field and colors the corresponding region of the shape using a probability-weighted blend.

Our system demonstrates the ability to reason about where to place non-obviously related concepts on an input 3dshape, such as adding clothing to a bare 3d animal model.

Authors

Dale Decatur, Itai Lang, Rana Hanocka

Related Topics

Neural network

Pre-trained vision-language model

Soft-selection operator

Neural optimization formulation

3d modeling workflows

Introduction

Semantic localization of regions on 3d meshes is an important problem in computer graphics and vision with broad applications.

One such application is the incorporation of semantic information into the 3d modeling process.

We propose, a method for automatically localizing semantic regions on a shape based on only a text description.

We optimize the weights of a neural network to produce probabilities that are used to color a given 3d shape in accordance with the specified text.

We leverage a pre-trained vision-language model to guide the neural optimization towards the text-specified region.

Our system contextualizes the text prompt and the corresponding shape region using the network-predicted probabilities.

Result

We present a technique for semantic regions on meshes using text as input, without any 3d datasets or 3d pre-training.

During neural optimization, our neural network infers a which we use to blend the highlight color onto the mesh.

The network-predicted probabilities are general, and provide a soft-segmentation which we show can be used for a variety of different applications (and).

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

3D Highlighter

3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions

We present 3d highlighter, a technique for localizing semantic regions on a mesh using text as input.

Our method contextualizes the text description using a neural field and colors the corresponding region of the shape using a probability-weighted blend.

Our system demonstrates the ability to reason about where to place non-obviously related concepts on an input 3dshape, such as adding clothing to a bare 3d animal model.

Authors

Dale Decatur, Itai Lang, Rana Hanocka

Related Topics

Neural network

Pre-trained vision-language model

Soft-selection operator

Neural optimization formulation

3d modeling workflows

Introduction

Semantic localization of regions on 3d meshes is an important problem in computer graphics and vision with broad applications.

One such application is the incorporation of semantic information into the 3d modeling process.

We propose, a method for automatically localizing semantic regions on a shape based on only a text description.

We optimize the weights of a neural network to produce probabilities that are used to color a given 3d shape in accordance with the specified text.

We leverage a pre-trained vision-language model to guide the neural optimization towards the text-specified region.

Our system contextualizes the text prompt and the corresponding shape region using the network-predicted probabilities.

Result

We present a technique for semantic regions on meshes using text as input, without any 3d datasets or 3d pre-training.

During neural optimization, our neural network infers a which we use to blend the highlight color onto the mesh.

The network-predicted probabilities are general, and provide a soft-segmentation which we show can be used for a variety of different applications (and).

Read the Paper

Read Paper

Home

Discover

Bookmarks

◐ Recommended Follows

◐ Latest Activity