New relica comprehensive observation platform for every engineer, announced that it is integrating its platform with NVIDIANIM inference microservices to reduce the complexity and cost of developing, deploying, and monitoring generative artificial intelligence (GenAI) applications. Now customers can benefit New Relic AI monitoring to achieve end-to-end AI stack visibility for applications built with NVIDIA NIM, all with simplified configuration and while ensuring data security. This complements the hearty security features and ease of operate of self-hosted models on NVIDIA NIM, which accelerates the delivery of generative AI applications. Together, Novel Relic integrated with NVIDIA NIM can lend a hand customers deploy AI faster and achieve faster ROI.
Observability is crucial to implementing cost-effective and proficient models
Organizations are rapidly adopting generative AI to improve digital experiences, escalate productivity and escalate revenue. Gartner predicts that by 2026, more than 80% of enterprises will operate GenAI or deploy GenAI applications. Rapid implementation and faster ROI are crucial for organizations to gain market advantage, and observability is key. It offers a holistic, real-time view of the AI application stack – across services, infrastructure and the AI layer – to ensure proficient, reliable and cost-effective operation.
Novel Relic accelerates ROI for AI applications built with NVIDIA NIM
AI applications can complicate technology stacks, escalate security concerns, and be cost-prohibitive. Novel Relic AI monitoring provides comprehensive visibility into your AI stack with key metrics around throughput, latency and cost, while ensuring data privacy. It also tracks request flows between services and models to understand the inner workings of AI applications. Novel Relic extends its detailed monitoring to NVIDIA NIM, supporting a wide range of AI models, including Databricks DBRX, Gemma by Google, Llama 3 by Meta, Phi-3 by Microsoft, Mistral Immense and Mixtral 8x22B, and Arctic by Snowflake. This helps organizations reliably deploy AI applications built with NVIDIA NIM, accelerate time to market and improve return on investment.
Key features and operate cases for AI monitoring include:
- Full visibility of your AI stack: Detect issues faster with a holistic view of applications, NVIDIA GPU-based infrastructure, AI layer, response quality, token counts, and APM gold signals.
- Deep analytics tracking every response: Fix performance and quality issues such as bias, toxicity, and hallucinations by tracking the entire AI response lifecycle
- Model inventory: Easily isolate model-related performance, error, and cost issues by tracking key metrics across all NVIDIA NIM inference microservices in one place
- Model comparison: Compare the performance of NVIDIA NIM inference microservices running in production in a single view to optimize model selection based on infrastructure and user needs.
- Deep GPU analytics: Analyze critical accelerated processing metrics such as GPU utilization, temperature, and performance states; understand context and solve problems faster.
- Increased data security: In addition to the security benefits of NVIDIA’s self-hosted model, Novel Relic allows you to exclude monitoring of sensitive data (PII) in AI requests and responses.
Novel Relic deepens its ecosystem of 60+ AI integrations with NVIDIA
This integration follows the recent addition of Novel Relic to NVIDIA AIOps Partner Ecosystem. Leveraging NVIDIA AI accelerated computing, Novel Relic combines observability and artificial intelligence to streamline IT operations and accelerate innovation with machine learning and a generative AI assistant, Novel Relic AI. Novel Relic offers the most comprehensive observability solution with over 60 AI integrations, including NVIDIA GPUs and NVIDIA Triton Inference Server.