LLaVANeXT

The LLaVANeXT class provides A wrapper for the LLaVANeXT model, which answers questions about visual media (images/videos) using the LLaVANeXT framework.

boolean

default:"True"

If True, inference call is run on the local VM, else offloaded onto GRID-Cortex. Defaults to True.

This model is currently not available via Cortex.

np.ndarray

The input RGB image of shape

(M,N,3)

str

The path to the input video.

str

required

The question to answer about the media.

str

The response to the prompt.

from grid.model.perception.vlm.llava_next import LLaVANeXT
car = AirGenCar()

# We will be capturing an image from the AirGen simulator 
# and run model inference on it.

img =  car.getImage("front_center", "rgb").data

model = LLaVANeXT(use_local = True)
result = model.run(rgbimage=img, prompt=<prompt>)
print(result)

License
Source

This code is licensed under the Apache 2.0 License.

⌘I

from grid.model.perception.vlm.llava_next import LLaVANeXT
car = AirGenCar()

# We will be capturing an image from the AirGen simulator 
# and run model inference on it.

img =  car.getImage("front_center", "rgb").data

model = LLaVANeXT(use_local = True)
result = model.run(rgbimage=img, prompt=<prompt>)
print(result)

Get Started

Open GRID

GRID Enterprise

Robot API

Simulation

AI Layer

Deployment

Data Generation Pipelines

FAQ