There's been a string of random occurrences of the following error in the SQL Server Agent scheduled jobs lately that I have been unable to find a solution to.
The error occurs infrequently, but usually once a week for a daily scheduled job, but in any number of different jobs and not always the same one. Each job shares the fact that it executes an SSIS package from the same server that is running the job. It also always runs for almost exactly 30 seconds elapsed time, which I guess is the timeout threshold. I'm not sure why it would timeout if the server is just connecting to its own SSIS catalog. Also of note is that it never actually gets to the point where it executes the SSIS package, and this occurs regardless of which package is trying to be executed.
During my research I came across many people suggesting that simply updating SQL Server 2012 to the latest CU* or SP2 would solve the problem. However, upgrading the server to SP2 has not.
One solution tried (which admittedly was ugly) was to simply have a single retry upon failure of the job step, which actually did solve the problem in about 30% of the cases.
I would welcome anyone who has experience with this error, or anyone who has any suggestions.
The error message is as follows:
Date 16/07/2014 6:00:11 AM
Log Job History ({$jobname})
Step ID 1
Server {$productionserver}
Job Name {$jobname}
Step Name {$stepname}
Duration 00:00:31
Sql Severity 0
Sql Message ID 0
Operator Emailed
Operator Net sent
Operator Paged
Retries Attempted 0
Message
Executed as user: {$user}.
Microsoft (R) SQL Server Execute Package Utility Version 11.0.5058.0 for 64-bit Copyright (C) Microsoft Corporation. All rights reserved.
Started: 6:00:11 AM Failed to execute IS server package because of error 0x80131904.
Server: {$productionserver},
Package path: {$packagepath},
Environment reference Id: NULL.
Description: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Source: .Net SqlClient Data Provider
Started: 6:00:11 AM Finished: 6:00:42 AM
Elapsed: 31.122 seconds. The package execution failed. The step failed.
Try this:
And this
Looks like its a known bug.
Check what else is/was running on the instance at the time of the package failures (e.g. a database integrity check or similarly intensive operation).
The SQL Agent is timing out talking to its own SSIS catalog (a 30 second timeout). It's not actually executing the packages, so it's nothing to do with the packages themselves and everything to do how busy the instance is at the time of the execution.
(Answering this question since it comes up in a Google search)
I know this is an older question. but I'm having the same problem and this doesn't have an accepted answer.
The job fails in 1.5 seconds so I believe it is NOT a timeout issue.
I can confirm 0x80131904 is (or can be) a permissions issue. I had my SSIS package running under a SQL Agent job just fine with sysadmin and network admin privileges. when i switched it to an account with fewer permissions, i get this error.
For me, the problem was because i was not assigning permissions in all the correct places. I already set Read/Execute permissions in the Project Properties. Then (this is the step I didn't do) I had to assign Read permissions on the folder containing Projects and Environments.
Hope this helps someone.
We have experienced this error when attempting to start several SSIS packages at the same instant. Service packs were supposed to fix it, but have not. We have implemented a staggered schedule for SSIS packages so only one package is starting at any given moment.
We also experienced the same bug. As a workaround, we created the following stored procedure. If you put this into a job that runs every f.e. 10 minutes, it makes sure that if there are random failures, the job gets restarted continuously until you reach an occurence without timeout failure.
USE [msdb]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[usp_StartTimedOutJob]
AS
DECLARE @jobid NVARCHAR(100)
, @jobname NVARCHAR(250)
, @stepname NVARCHAR(250)
, @varMail VARCHAR(MAX)
DECLARE cJobs CURSOR FOR
-- CTE selects all jobs that are currently not running and orders them by most recent
WITH CTE_NotRunning AS (
SELECT S.job_id
, S.step_name
, S.[message]
, rownum = ROW_NUMBER() OVER (PARTITION BY S.job_id ORDER BY S.run_date DESC, S.run_time DESC)
FROM msdb.dbo.sysjobhistory AS S
LEFT OUTER JOIN (SELECT DISTINCT ja.job_id
FROM msdb.dbo.sysjobactivity ja
LEFT JOIN msdb.dbo.sysjobhistory jh ON ja.job_history_id = jh.instance_id
JOIN msdb.dbo.sysjobs j ON ja.job_id = j.job_id
JOIN msdb.dbo.sysjobsteps js
ON ja.job_id = js.job_id
AND ISNULL(ja.last_executed_step_id,0)+1 = js.step_id
WHERE
ja.session_id = (
SELECT TOP 1 session_id FROM msdb.dbo.syssessions ORDER BY agent_start_date DESC
)
AND start_execution_date is not null
AND stop_execution_date is NULL) AS R
ON S.job_id = R.job_id
WHERE R.job_id IS NULL)
-- only select the jobs into the cursor set for which the most recent job had a timeout issue
SELECT job_id
, step_name
FROM CTE_NotRunning
WHERE [message] LIKE '%0x80131904%time%out%' -- error message that corresponds to timed out jobs, error code: 0x80131904
AND rownum = 1
OPEN cJobs
FETCH NEXT FROM cJobs
INTO @jobid, @stepname
WHILE @@FETCH_STATUS = 0
BEGIN
-- for each of the timed out jobs in the cursor, start the job again from the step that caused the timeout
SET @jobname = (SELECT [name] FROM msdb.dbo.sysjobs WHERE job_id = @jobid)
EXECUTE dbo.sp_start_job @job_id = @jobid, @step_name = @stepname
END
CLOSE cJobs
DEALLOCATE cJobs
GO
I had this exact same issue. SQL Agent was running SSIS Jobs perfectly fine then suddenly I came across this error. Spent about an hour looking for a fix online. Found out the server admin had installed new windows updates.
I simply restarted the Server (which hosts the SSIS catalog and SQL Server/Agent). After server restart jobs ran fine again.
Hope server restart works for the next person that goes through this.
Sometimes this kind of error occurs when the package is deployed twice under SQL Integration Service Catalogs. You also may have changed the package name but there are other related auto-generated configurations are unique like the Environment reference Id and others .
So if you have a scheduled job, you will need to create a new one and point it to the .
Good Luck
User contributions licensed under CC BY-SA 3.0