Sending Airflow DAG Errors to VictorOps

By WikiExplain on 2023-03-14 · 3 mins read

If you are running your Airflow DAGs(Directed Acyclic Graph) in Google Cloud Composer , you would think you could use a GCP Monitoring Metrics. However, at the time of this writing, there is not an existing default metric we could use for DAGs or Tasks errors. Don't worry there is an easy way around it, leveraging Airflow natural support for email notification.

Setting up Airflow with VictorOps and Slack

Airflow Email Notification System

As show in the image above, we need to set up an email notification system. Luckily for us, we do not need to build anything only configure the different pieces of the puzzle. Follow the steps below to configure the corresponding service as needed.

I. Integrate Slack to VictorOps

Note: The official guide detailed requirements and helpful tools if you looking for a more in depth instructions



  • Navigate to VictorOps web portal
  • Select Integrations
  • Select 3rd Party Integrations
  • Search for Slack
  • Click Enable Integration
  • Enter the desired workspace and log in
  • Follow the instruction and authorize the application
  • Select a default Slack channel
  • II. Create VictorOps Alert Routing Key

Note: Navigate to Settings >> Routing keys and verify if you already have a Slack channel routing key



VictorOps allows you to send emails from your monitoring service or email provider to a dedicated VictorOps address. This way you will be creating, acknowledging, or resolving incidents in your VictorOps wall, I mean timeline :)



  • Log into VictorOps web portal
  • Select Integrations
  • Search Email Generic
  • Click Enable Integration button
  • Replace $routing_key with your Routing Key from the previous step

IV. Configure Airflow in Google Composer with SendGrid

  • Log into your Google Cloud console account
  • Select START WITH THE FREE PLANto create a SendGrid account
  • Fill up the form in the popped up window
  • Select Go Back to Google
  • Select Manage API keys on SendGrid website
  • Type a name for the key
  • Select Web API
  • Select Create API Key
  • Copy the generated key
  • Go back to your Cloud Composer environment
  • Select Edit
  • Enter SENDGRID_API_KEY for the name
  • Pasted the generated SendGrid key for the value
  • Select ADD ENVIRONMENT VARIABLES to add another variable
  • Enter SENDGRID_MAIL_FROM for the name and your email from for the value. It could be the same one you use to create the SendGrid account or something such as noreply-composer@.
  • Select Save and grab a drink :)

Configure Airflow


from airflow.operators.email_operator import EmailOperator
def report_failure(context):
subject = 'Airflow critical: {{ti}}'
html_content = (
"""Send custom email alerts."""
return EmailOperator(
  • Configure the fall back property in the DAG arguments.
default_args = {
'on_failure_callback': report_failure,
'email_on_failure': False,
'email_on_retry': False}

Configure Airflow with AWS

  1. Enable AWS SES service
  2. Select SMTP Settings in the left side
  3. Select Create My SMTP Credentials
  4. Copy your credentials or click the "Download Credentials"
  5. Go to your code base and open the file airflow.cfg and replace the following properties with your downloaded/copied AWS credentials and you are done :)
smtp_starttls = True
smtp_ssl = False
smtp_user = SES-user-from-previous-step
smtp_password = SES-password-from-previous-step
smtp_port = 587
smtp_mail_from =

\ Note: Port will be provided by the SMTP provider typically is 25, 587

Configure Airflow with SendGrid SMTP Relay

  • Sign in to SendGrid
  • Select under Integrate using our Web API or SMTP relay
  • Select Choose under SMTP Relay
  • Type your API key name
  • Select Create Key
  • Select Next: Verify Integration
  • Verify the properties
  • Open airflow.cfg in your code repo
  • Replace properties with your smtp sendgrid generated values.
  • Set smtp_host to t
  • Set your smtp_user to apikey.
  • Set smtp_password to the API key generated in your SMTP account.
  • Set smtp_port to 587
  • Set smtp_mail_from to your account email

Configure Airflow with Gmail

  • Login to Gmail
  • Go to App passwords in your account security section
  • Select App
  • Select your device
  • Generate your password
  • Make sure you copy the generated password in somewhere you won't be able to see it again
  • Open airflow.cfg
  • Set smtp_host to
  • Set smtp_user to your Gmail address
  • Set smtp_password to the generated 16 digits App passwords
  • Set smtp_port to 587
  • Set smtp_mail_from to your Gmail address

Show me the code!

smtp_host =
smtp_starttls = True
smtp_ssl = False
smtp_user =
smtp_password = Your-[App passwords](
smtp_port = 587
smtp_mail_from =

Another option can be to configure Airflow with esuite Relay