Owns access to language model inference for the current tenant, registered for GraalJS scripts as llmManager. It merges models offered by any statically configured LlmProvider with those contributed by active apps that implement LlmProviderApp, dispatches invokeInference calls to the matching provider, and notifies any registered LlmInferenceListener before and after each call so usage can be recorded or audited. It also stores the tenant's consent preference for sending data to third party language model providers.

Group: Managers

---

Properties

---

Methods

getAvailableModels() · getModelByName(String name) · getModelForTier(String tier) · invokeInference(InferenceParameters parameters) · getUsage(String provider, Date startDate, Date finishDate) · inferenceParametersBuilder() · getConsentPreference() · hasUserConsented() · setConsentPreference(String value) · newJsonSchemaResponseFormatBuilder(String schemaName)

getAvailableModels()

Returns: List<LlmModel>

Finds the language models available to the current tenant, combining any statically configured providers with providers contributed by active apps, including both models available to every tenant and models scoped to this tenant only such as fine tuned models. Within a single request the result is cached on the RequestContext, so repeated calls do not re-query the active apps.

getModelByName(String name)

Returns: LlmModel

Finds an available model by its name, searching the models returned by getAvailableModels. Logs a warning and returns null if no model with the given name is available to the current tenant.

getModelForTier(String tier)

Returns: LlmModel

Resolves what a caller wants a model <i>for</i> into the best model available to this tenant for that job. <p> Callers should prefer this over naming a model outright. A named model is correct only until the next release, and it is correct only for tenants that happen to have it enabled; a tier stays correct through both. See {@link LlmModelTier}. <p> Where the tenant has none of the tier's preferred models, this degrades to any available model that can do agentic work - function calling and reasoning - rather than failing the caller's run. Only a tenant with no models at all is an error, because there is nothing to degrade to.

invokeInference(InferenceParameters parameters)

Returns: Object

Passes the body to the specified LLM model and returns the raw response

getUsage(String provider, Date startDate, Date finishDate)

Returns: List<List>

Finds the usage records logged by the named provider between the given dates. The record shape is provider-specific.

inferenceParametersBuilder()

Returns: InferenceParametersBuilder

Creates a new, empty InferenceParametersBuilder for assembling the InferenceParameters passed to invokeInference.

getConsentPreference()

Returns: String

The current tenant's stored preference for allowing data to be sent to third party language model providers, read from the admin-lib app setting for the current organisation.

hasUserConsented()

Returns: boolean

Whether the current tenant has consented to sending data to third party language model providers, based on getConsentPreference being set to "yes".

setConsentPreference(String value)

Returns: void

Stores the current tenant's preference for allowing data to be sent to third party language model providers, as the admin-lib app setting checked by getConsentPreference and hasUserConsented.

newJsonSchemaResponseFormatBuilder(String schemaName)

Returns: JsonSchemaResponseFormatBuilder

Creates a new JsonSchemaResponseFormatBuilder for the named schema, used to build a response_format map that forces a model's reply into a specific JSON structure.