20 min read

The Rise of AI-Assisted IT Automation: Benefits, Risks, and Human Oversight

ai assisted automation.webp

AI-assisted IT automation is changing how managed service providers (MSPs) and internal IT teams monitor infrastructure, troubleshoot endpoints, investigate incidents, and manage routine maintenance.

Traditional IT automation follows predefined instructions. AI introduces another capability: interpreting operational information, generating recommendations, and interacting with management tools to help technicians complete complex tasks.

This creates opportunities to reduce repetitive work and improve operational efficiency. However, allowing AI to influence or execute administrative actions also introduces risks involving inaccurate recommendations, excessive permissions, sensitive data exposure, and unintended system changes.

The central challenge is determining which activities AI should assist with, which actions can be safely automated, and which decisions must remain under human control.

For IT teams, successful AI adoption requires more than capable technology. It requires reliable operational data, established automation procedures, enforceable security boundaries, and meaningful human oversight.

What Is AI-Assisted IT Automation?

AI-assisted IT automation uses artificial intelligence to support IT management activities, including monitoring, troubleshooting, incident investigation, system maintenance, and workflow development.

Unlike conventional automation, which primarily follows predefined rules, AI can interpret context and generate recommendations based on the information available to it.

Three related approaches are worth distinguishing:

  • Traditional IT automation: Executes predefined actions based on established triggers, conditions, and workflows.
  • AI-assisted automation: Uses artificial intelligence to analyze information, recommend actions, generate scripts, or help technicians operate management tools.
  • Agentic AI: Uses AI systems capable of planning and coordinating multiple steps, invoking tools, and evaluating results with varying degrees of autonomy.

These approaches can work together.

For example, a monitoring platform may detect a recurring service failure. An AI assistant can review relevant logs and suggest a possible cause. A technician can verify that diagnosis before authorizing an established remediation workflow.

In this arrangement, AI supports investigation and decision-making, while conventional automation provides predictable execution.

The important distinction is that AI intelligence and execution authority are separate capabilities. An AI system can provide useful analysis without having permission to modify production infrastructure.

How AI Is Changing IT Operations

As AI assistants gain access to command-line interfaces, APIs, and operational tools, they can support activities that previously required technicians to manually collect and interpret information from multiple systems.

Several applications are particularly relevant to MSPs and internal IT departments.

AI-Assisted Monitoring and Incident Investigation

Monitoring platforms generate alerts based on device health, system performance, service availability, and other operational conditions.

AI can help technicians interpret these alerts by organizing related events, summarizing relevant telemetry, and identifying potential relationships between symptoms.

For example, an application server experiencing repeated service failures might also show elevated memory consumption and unusual process activity.

An AI assistant could summarize these observations and suggest diagnostic steps.

However, identifying a correlation does not establish a root cause. AI recommendations must be checked against available telemetry and diagnostic evidence before remediation.

Endpoint Troubleshooting

Endpoint troubleshooting often involves reviewing operating system logs, running processes, configuration settings, and previous maintenance activity.

AI can assist by:

  • Summarizing Windows Event Viewer entries or Linux system logs.
  • Explaining error messages and recurring service failures.
  • Comparing configurations against documented standards.
  • Generating read-only diagnostic commands.
  • Suggesting relevant troubleshooting procedures.

These capabilities can reduce the time technicians spend collecting and interpreting information.

The benefit depends on the completeness and accuracy of the underlying data. An AI assistant cannot reliably diagnose information it has not received.

Script Generation and Workflow Development

Scripting remains a major component of IT automation. Technicians commonly use PowerShell, Bash, Python, and other languages to manage endpoints and perform maintenance.

Generative AI can help draft scripts, explain existing code, identify possible errors, and translate operational requirements into automation specifications.

For example, a technician could request a PowerShell script that inventories installed applications and exports the results.

The generated script may provide a useful starting point, but it still requires validation.

Reviews should consider target selection, operating system compatibility, execution privileges, error handling, and unintended side effects.

AI-generated scripts should be treated as unreviewed code, not automatically trusted production procedures.

Patch Management and Maintenance

AI can assist with organizing update information, summarizing vendor advisories, and identifying potentially affected systems using verified inventory data.

It can also help technicians prepare deployment plans and review reported compatibility issues.

However, patch prioritization requires more than interpreting an advisory.

Teams must consider vulnerability severity, exploitability, asset exposure, operational dependencies, maintenance windows, and deployment testing.

AI-assisted analysis can complement established patch management workflows, but it should not bypass approved deployment controls.

Incident Documentation and Knowledge Management

AI can transform troubleshooting notes, command results, and incident timelines into structured documentation.

For MSPs, this can make it easier to transfer work between technicians and maintain consistent customer records.

AI can also help retrieve relevant procedures from an organization's approved knowledge base.

However, generated documentation must accurately represent what occurred. An AI summary should not be treated as proof that a particular action was performed or that an incident was resolved.

AI-Assisted Automation vs. Traditional Automation: Which Should IT Teams Use?

AI is not inherently a better approach for every automated task.

Traditional automation is often more appropriate when the required conditions and actions are predictable. AI becomes useful when technicians must interpret information, investigate unfamiliar conditions, or prepare new procedures.

IT activity

Recommended approach

Reason

Summarizing incident logs

AI assistance

Helps interpret large amounts of information

Running scheduled maintenance

Traditional automation

Executes predictable, tested procedures

Investigating an unfamiliar service failure

AI with technician validation

Requires interpretation and diagnostic judgment

Drafting a remediation script

AI assistance with code review

Accelerates development without trusting untested code

Restarting a known failed service

Bounded traditional automation

Can follow validated conditions and safeguards

Deploying a new script across production servers

Human-approved execution

Carries broader operational risk

Preparing incident documentation

AI assistance with verification

Reduces administrative effort while preserving accuracy

The strongest approach is often a combination.

AI helps technicians understand an unfamiliar situation or develop a solution. Once a procedure is tested and its conditions are understood, conventional automation can execute it consistently.

This prevents organizations from introducing unnecessary AI dependencies into tasks that existing automation already handles reliably.

The Operational Benefits of AI-Assisted IT Automation

AI-assisted automation can improve efficiency, but the benefits must be evaluated across the entire operational workflow.

Faster Investigation and Troubleshooting

AI can reduce the manual effort required to gather information, interpret logs, and prepare diagnostic procedures.

Technicians can use the resulting analysis to determine which actions require further investigation.

The potential improvement is greatest when AI provides relevant, accurate information without creating substantial additional review work.

Improved Technician Capacity

Reducing repetitive administrative and investigative work can allow technicians to spend more time on complex incidents, customer support, and infrastructure improvements.

For MSPs, this may increase the amount of work existing staff can manage.

However, recovered technician time does not automatically represent reduced payroll expenses.

Financial benefits depend on whether the additional capacity leads to measurable outcomes, such as reduced overtime, avoided hiring, or additional profitable service delivery.

More Consistent Operational Procedures

AI can assist with generating standardized troubleshooting notes, script drafts, and incident documentation.

When paired with approved runbooks and established automation workflows, this can help reduce inconsistencies between technicians.

Consistency still depends on accurate documentation and appropriate validation.

Better Utilization of Existing Management Tools

AI can help technicians discover and reuse existing scripts, monitoring data, and automation workflows.

Rather than creating a new solution for every recurring issue, teams can use AI to identify an applicable established procedure.

This can extend the value of existing IT management investments without requiring every workflow to become autonomous.

Measuring the Actual Efficiency Gain

Consider an illustrative MSP workflow involving 100 incident investigations per month.

Before introducing AI assistance, each investigation requires an average of 30 minutes.

After introducing AI, the initial investigation takes 18 minutes, with an additional 5 minutes spent reviewing and correcting AI output.

Illustrative monthly comparison:

Measurement

Manual workflow

AI-assisted workflow

Investigations per month

100

100

Initial investigation time

30 minutes

18 minutes

Review and correction time

Included in original workflow

5 minutes

Total time per investigation

30 minutes

23 minutes

Monthly technician effort

50 hours

38.3 hours

In this example, AI assistance recovers approximately 11.7 technician hours per month, representing a 23.3% reduction in investigation effort.

These figures are hypothetical and are not industry benchmarks or Level customer results.

The example also excludes implementation, training, licensing, and other operating costs.

Recovered time becomes a financial benefit only when its actual economic value can be established.

Organizations should measure performance using their own incident records, technician effort, rework rates, and operational costs.

The Security Risks of AI-Assisted IT Automation

The consequences of AI errors become more significant when AI systems receive permission to interact directly with production infrastructure.

An assistant that summarizes logs presents a different risk from an agent capable of executing administrative commands across an endpoint fleet.

The OWASP Top 10 for LLM Applications 2026 and OWASP Top 10 for Agentic Applications 2026 provide relevant guidance for understanding these risks.

Incorrect Recommendations and Unreliable Conclusions

Generative AI can produce technically plausible but incorrect explanations.

For example, an AI assistant might attribute application failures to insufficient memory even when the actual problem involves network connectivity.

Following an incorrect recommendation can waste investigation time or introduce additional issues.

Safeguard: Require evidence-backed diagnostic conclusions and validate recommendations against actual system information before making changes.

Prompt Injection and Untrusted Data

AI systems may process support tickets, retrieved documentation, log files, and other information that cannot be fully trusted.

An attacker could embed malicious instructions within these sources and attempt to influence the AI system's actions.

This is known as prompt injection.

For example, malicious content within a retrieved support document might attempt to persuade an AI assistant to execute an unrelated administrative command.

According to OWASP's prompt injection guidance, the underlying problem is that instructions embedded in untrusted content can influence model behavior.

Safeguard: Treat retrieved content as data, independently enforce tool permissions, and prevent untrusted information from expanding an agent's authorized scope.

Excessive Administrative Permissions

AI agents connected to management tools may inherit powerful administrative capabilities.

If an agent can execute arbitrary scripts or modify large groups of endpoints, an incorrect decision could affect numerous systems.

This risk is particularly important in MSP environments where different customers must remain operationally separated.

Safeguard: Apply least-privilege access, restrict tool capabilities, and enforce tenant and device boundaries through the underlying management platform.

Sensitive Data Exposure

Monitoring systems and management tools can contain device inventories, diagnostic logs, user information, and configuration details.

Sending this information to external AI services creates additional data-handling considerations.

Organizations should establish which information can be transmitted, which providers are approved, how data is retained, and which contractual or regulatory requirements apply.

Safeguard: Implement approved data-access and retention policies before connecting AI services to sensitive operational systems.

Partial Execution and Incomplete Changes

A multi-step automation may successfully perform its initial operations but fail before completing the remaining steps.

This can leave systems in inconsistent states.

For example, a workflow might stop a service and modify configuration files but fail before restarting the service.

AI-generated workflows may also create or modify resources through several separate operations that are not transactional.

Safeguard: Design workflows with defined failure behavior, state verification, and recovery procedures. Do not assume that a failed operation automatically reverses earlier changes.

Repeated Remediation and Cascading Failures

An automation that repeatedly attempts the same ineffective remediation can worsen an incident.

Repeated service restarts, unnecessary system changes, or resource-intensive diagnostic scripts may increase disruption.

The risk grows when actions are executed across multiple endpoints.

Safeguard: Establish execution limits, retry controls, failure thresholds, and mechanisms to stop further actions.

Stale Data and Incorrect Target Selection

AI recommendations depend on the information available to the system.

An assistant using outdated inventory data could select a device that has changed roles or recommend a configuration based on obsolete information.

A similarly named device in another customer environment could also create targeting mistakes.

Safeguard: Verify current device identities, relevant inventory details, and execution targets immediately before high-impact actions.

Limited Accountability and Auditability

When an AI system performs multiple operations, administrators need to reconstruct which actions occurred and why.

A natural-language summary is not sufficient by itself.

Operational records should identify the initiating request, execution identity, selected targets, approved actions, actual commands, results, and errors.

Safeguard: Maintain auditable execution records and retain appropriate system-generated evidence.

Why Human Oversight Is Essential in AI Automation

Human oversight ensures that AI-assisted operations remain subject to appropriate organizational control.

However, oversight does not mean requiring technicians to manually approve every routine task.

It means applying controls based on the potential impact of an action.

The NIST AI Risk Management Framework's Generative AI Profile emphasizes risk-based governance, documentation, monitoring, and appropriate human involvement in generative AI systems.

For IT operations, a useful starting point is distinguishing three levels of oversight.

Level 1: AI Recommendations With Human Execution

AI analyzes information and recommends actions but cannot directly modify production infrastructure.

Suitable activities include:

  • Incident investigation.
  • Read-only diagnostics.
  • Script drafting.
  • Troubleshooting recommendations.
  • Incident documentation.

The technician independently decides whether to execute the proposed action.

This provides an opportunity to evaluate AI reliability before granting administrative capabilities.

Level 2: Human-Approved AI-Assisted Execution

AI prepares a proposed operation, but execution requires explicit authorization.

This is appropriate for newly generated scripts, production configuration changes, and other potentially disruptive activities.

Approval should include the actual commands or complete workflow specification, execution identity, affected devices, and expected consequences.

An approval prompt that merely says "Apply recommended changes" provides insufficient information for meaningful review.

Level 3: Bounded Automated Execution

A validated workflow can execute without requiring individual approval for every occurrence, provided its operating conditions and permissions remain within approved boundaries.

Examples include:

  • Scheduled maintenance procedures.
  • Tested monitoring-triggered remediation.
  • Approved endpoint configuration checks.
  • Established service recovery workflows.

This approach is most appropriate for predictable tasks with defined conditions, limited execution scope, and acceptable failure behavior.

AI may assist with developing or investigating these workflows without controlling every execution decision.

How to Decide Which Actions Require Human Approval

Approval requirements should consider several operational factors.

1. Does the action modify production systems?

Read-only analysis generally presents less operational risk than changing configurations, installing software, or deleting files.

2. Can the action be reversed?

A reversible configuration change may be easier to recover from than deleting data or changing security credentials.

However, even a theoretically reversible action can interrupt services.

3. How many systems are affected?

Changing one test endpoint carries different consequences from executing the same command across hundreds of production devices.

4. Has the procedure been tested?

A previously validated script with known behavior generally presents less uncertainty than newly generated code.

5. Does the action cross security or customer boundaries?

Administrative activities affecting multiple customers, privileged accounts, or sensitive systems warrant stronger controls.

6. Can the organization verify and stop execution?

Safe automation requires visibility into execution results and a way to prevent additional actions when unexpected behavior occurs.

These considerations should determine approval requirements rather than relying solely on whether an action was proposed by AI.

Human Oversight Must Be Enforced Outside the AI Model

Instructing an AI assistant to request approval is not equivalent to enforcing approval.

An agent with unrestricted administrative permissions may still be capable of executing a command through its connected tools.

Effective controls should therefore exist in the authentication, authorization, workflow, or execution systems themselves.

Important safeguards include role-based access controls, explicit target restrictions, independently enforced approval gates, execution limits, and auditable records.

Organizations should also verify that the executed action matches the approved operation.

Human oversight is meaningful only when reviewers have sufficient information to make a decision and the technical environment enforces that decision.

Practical Example: AI-Assisted Disk Space Remediation

Consider an MSP managing production application servers.

A monitoring platform detects that a server's available disk space has fallen below its configured threshold.

The MSP wants to accelerate investigation without allowing AI to modify production systems without appropriate authorization.

Step 1: Monitoring Identifies the Issue

An existing monitoring rule generates an alert containing the affected server, storage utilization, and relevant device information.

Step 2: AI Reviews Diagnostic Information

An authorized AI assistant retrieves read-only information about disk usage, relevant logs, and available system context.

The assistant identifies a log directory consuming substantial storage and proposes further investigation.

The technician confirms whether the files are eligible for removal or rotation.

Step 3: AI Prepares a Proposed Remediation

The assistant provides:

  • The identified cause and supporting evidence.
  • The intended corrective action.
  • The exact commands or script.
  • The target server and execution privileges.
  • Potential risks and expected results.
  • Verification and recovery considerations.

No production change occurs at this stage.

Step 4: An Authorized Technician Reviews the Action

A technician reviews the proposed operation and confirms that it complies with the customer's requirements.

The technician authorizes the specific corrective action through the applicable management process.

Step 5: The System Executes and Verifies

The approved operation executes against the intended server.

Monitoring then checks disk utilization and relevant service health.

If the procedure fails, the workflow should report the failure rather than automatically repeating potentially disruptive actions without appropriate limits.

Step 6: The Team Documents the Outcome

The incident record captures the original alert, investigation, approval, executed action, and verified result.

If the same problem recurs, the team can evaluate whether a tested, bounded automation would provide a reliable long-term solution.

This example demonstrates how AI-assisted investigation can improve troubleshooting without requiring unrestricted autonomous execution.

How MSPs Can Implement AI Automation Safely

Introducing AI into IT operations should follow a controlled process rather than immediately granting broad administrative access.

1. Identify Suitable Workflows

Start with activities that consume meaningful technician time but present limited operational risk.

Read-only investigation, alert summaries, documentation, and script drafting are useful starting points.

2. Establish Data and Access Boundaries

Define the information AI systems may access and the management operations they may perform.

Separate read-only investigation permissions from administrative execution privileges.

MSPs should preserve customer-level access boundaries.

3. Test AI Recommendations

Evaluate AI output using representative operational scenarios.

Measure whether recommendations are accurate, supported by evidence, and useful to technicians.

Unreliable recommendations should not automatically progress into executable workflows.

4. Validate Scripts and Automation Specifications

Generated scripts and workflow definitions should undergo review and testing.

Verify command behavior, permissions, target selection, error handling, and the potential consequences of partial execution.

5. Define Approval and Failure Controls

Establish which activities require explicit technician authorization.

Configure retry limits, failure handling, execution boundaries, and escalation procedures.

Where supported, preview proposed workflow changes before applying them.

6. Monitor the Complete Operational Workflow

Track not only how quickly AI produces an answer but also the time required to review, execute, verify, and correct its recommendations.

Relevant metrics include incident resolution time, technician effort, recommendation accuracy, failed actions, and service quality.

7. Expand Automation Based on Verified Results

Increase execution authority only after workflows demonstrate acceptable reliability under appropriate controls.

A successful pilot does not automatically justify extending an AI agent's permissions across an entire customer environment.

How Level Supports AI-Assisted IT Automation

AI-assisted IT operations require access to reliable endpoint information, established management capabilities, and controlled execution tools.

Level provides remote monitoring and management capabilities that MSPs and internal IT departments can use to monitor endpoints, execute scripts, manage updates, and automate operational procedures.

Connecting External AI Agents Through Level CLI

In 2026, Level introduced the ability to connect compatible external coding agents to its management environment through Level CLI.

The CLI allows technicians and compatible coding agents to interact with supported Level resources, including devices, alerts, scripts, automations, and monitoring policies.

AI-assisted workflows can help technicians investigate endpoint issues, inspect relevant management information, and prepare scripts or automation specifications.

Importantly, Level does not host the underlying AI model or provide model tokens through this integration. Organizations supply their own compatible coding agent and model provider.

According to Level's Bring Your Own Agent documentation, connected agents use CLI commands with the permissions of the active Level session.

This makes permission management and command review important.

The documentation recommends reviewing proposed commands, device targets, generated scripts, and automation specifications before execution.

Level CLI also provides dry-run functionality for supported automation and monitoring policy changes.

However, dry-run capabilities do not make every operation harmless. Creating or applying some workflow specifications involves multiple changes that are not transactional, meaning failures may leave partially applied configurations.

Additionally, certain administrative CLI operations can execute without a separate confirmation prompt.

Organizations should therefore avoid assuming that connecting an AI agent automatically creates an approval-controlled environment.

Combining AI Assistance With Level Automations

Level's automation platform supports scheduled and event-driven workflows for endpoint management and maintenance.

Its scripting capabilities allow technicians to execute supported scripts across managed devices.

These established management functions can complement AI-assisted investigation.

For example, an AI assistant may help identify the cause of a recurring service failure and draft a remediation procedure.

A technician can then review and test the procedure before implementing it through an appropriate Level automation.

For workflows requiring technician authorization, Level provides a Wait for Approval action that pauses an automation until an authorized technician approves or rejects it.

This is a useful control, but it has operational limitations.

Pending approvals do not automatically expire, and the action does not automatically notify technicians when approval is required.

Teams must therefore establish a process for monitoring pending approvals and ensuring that reviewers understand the relevant action.

Level's approval feature also should not be confused with individually approving every patch before installation. Patch deployment uses separate controls, including update delays and exclusions.

The Role of RMM in AI-Assisted Operations

RMM provides the operational foundation for managing devices, monitoring conditions, executing maintenance procedures, and maintaining visibility into endpoint environments.

AI can assist technicians in interpreting that information and preparing actions.

The management platform remains responsible for providing the supported execution capabilities and enforcing its configured permissions and workflow controls.

This separation allows organizations to introduce AI assistance without replacing established endpoint management practices.

MSPs and internal IT departments can explore Level's management capabilities to evaluate how monitoring, scripting, automations, and CLI access fit into their operational requirements.

Final Thoughts

AI-assisted IT automation is expanding what technicians can accomplish with existing management tools.

By helping interpret information, develop scripts, and investigate incidents, AI can reduce repetitive effort and support more efficient IT operations.

However, the same capabilities introduce additional risks when AI systems receive administrative permissions or influence production changes.

The most effective approach is to combine AI-assisted analysis with tested automation procedures, enforceable access controls, reliable execution records, and human approval where the operational impact requires it.

Organizations should begin with measurable, low-risk use cases and expand automation authority only when the results and safeguards justify it.

The objective is not unrestricted automation. It is using AI to improve IT operations while retaining control over the systems, decisions, and outcomes that matter.

Frequently Asked Questions

1. What is the difference between AI-assisted IT automation and traditional IT automation?

Traditional IT automation executes predefined tasks based on established triggers and conditions. AI-assisted IT automation adds capabilities such as interpreting operational data, recommending actions, generating scripts, and interacting with management tools. The two approaches can complement each other, with AI supporting analysis while traditional automation handles predictable execution.

2. Can AI automate IT troubleshooting without human intervention?

AI can assist with or independently perform some troubleshooting activities, depending on its permissions and available tools. Read-only diagnostics and tested, narrowly scoped remediation procedures may be suitable for automated execution. High-impact or unfamiliar production changes generally require stronger validation, access controls, and human authorization.

3. What are the biggest security risks of AI-assisted IT automation?

Important risks include prompt injection, incorrect recommendations, excessive administrative permissions, sensitive data exposure, unintended system changes, and incomplete workflow execution. Organizations can reduce exposure through least-privilege access, independently enforced permissions, approved execution boundaries, testing, and audit records.

4. How can MSPs measure the effectiveness of AI-assisted IT automation?

MSPs can measure investigation time, incident resolution time, technician effort, recommendation accuracy, and automation failure rates. Measurements should include the time required to review and correct AI output. Recovered capacity should be distinguished from actual cost savings or additional profitability.