CoChat
Sign Up
Cited by 4Software System Performance and ReliabilityArtificial Intelligence ApplicationsAdversarial Robustness in Machine Learning

LLMOps, AgentOps, and MLOps for Generative AI: A Comprehensive Review

Open access
Joshi, Satyadhar·2025·International Journal of Computer Applications Technology and Research
DOI 10.7753/ijcatr1407.1001
Open this paper in CoChat — free account

Abstract

AgentOps is an emerging discipline focused on the operationalization, monitoring, and optimization of agentic AI systems, especially in the context of generative AI.This paper provides an overview of AgentOps, its relationship to MLOps and GenAIOps, and best practices for deploying and managing agentic AI in enterprise environments.This paper shows how the evolution happened form operational methodologies in AI, from traditional Machine Learning Operations (MLOps) to finally specialized Large Language Model Operations (LLMOps) and then to Generative AI Operations (GenAIOps).We examine the key challenges in operationalizing generative AI, including model monitoring, prompt management, agent debugging, and ethical considerations.Through a systematic review of over 100 recent articles and industry practices, we identify critical gaps in current operational approaches and summarize based on recent literature an integrated framework that combines MLOps, LLMOps, and AgentOps principles.We present case studies demonstrating successful implementations and provide recommendations for organizations transitioning to generative AI at scale.We delineate the distinct characteristics, challenges, and best practices associated with each stage, emphasizing how AgentOps extends prior concepts to manage the unique complexities of multi-agent systems, including monitoring, debugging, and secure lifecycle management.We provide a holistic overview of the current landscape and future directions for operationalizing advanced AI systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7753/ijcatr1407.1001

Cites

8 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • These frameworks give AI agents the ability to plan, remember past actions, reflect on their performance, pursue goals, and use external tools.

    “These frameworks give AI agents the ability to plan, remember past actions, reflect on their performance, pursue goals, and use external tools.”

    From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [26]

    The “AgentOps: The Next Frontier” section describes agents that plan, use memory and external tools, and pursue goals. It says AgentOps manages and monitors those agents, rather than giving them those abilities. Our copy does not establish that the frameworks enable agents to remember past actions or reflect on their own performance. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • LLM-based frameworks like LangChain, AutoGPT, BabyAGI, AutoGen, and OpenAgents show a clear trend toward building smarter, more independent AI systems.

    “The development of LLM-based frameworks like LangChain [26,31,78,80], AutoGPT [26,31,78], BabyAGI [31], AutoGen [80], and OpenAgents [26] shows a clear trend toward building smarter, more independent AI systems.”

    From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [26]

    The conclusion describes growing complexity and independence in AI systems, and the 'Operationalizing Generative AI: From MLOps to AgentOps' section describes a shift toward autonomous, reasoning-driven agents. Those passages support the general trend, but our copy does not identify LangChain, AutoGPT, or BabyAGI as examples of it; searches for those names found no matching passage. Page numbers were not supplied for these sections. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. "AgentOps Review & Alternatives (2025)," AI Agents Directory. https://aiagentsdirectory.com/agent/agentops, Jan. 2025.
  2. "AWS Workshops." https://workshops.aws/categories/MLOps.

Cited by

Sorted by citations

Related

Authors
Joshi, S.
Corresponding author
Joshi, Satyadhar
Venue
International Journal of Computer Applications Technology and Research
Publisher
Association of Technology and Science
Published
2025-06-27
ISSN
2319-8656 (ISSN-L)
DOI
10.7753/ijcatr1407.1001
OpenAlex
W4411716463
Cited by
4
References
9
Citations by year
2025: 4
FWCI
1.68
Type
journal article
Language
en
Open access
Locations
  • International Journal of Computer Applications Technology and Research (publishedVersion)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
LLMOps, MLOps, agentic AI, multi-agent systems, model monitoring, prompt management, agent debugging, AI ethics, AI lifecycle management, enterprise AI deployment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review