Skip to main content

Run Argo workflows

Install in your project

Argo needs to be installed in your project, contact the DSRI team to request it.

Install the argo client​

Argo 🦑 is a container native workflow engine for Kubernetes supporting both DAG and step based workflows.

Download and install the Argo client on your computer to start workflows on the DSRI.

On Ubuntu​

sudo curl -L -o /usr/local/bin/argo https://github.com/argoproj/argo/releases/download/v2.4.2/argo-linux-amd64
sudo chmod +x /usr/local/bin/argo

On MacOS​

brew install argoproj/tap/argo

On Windows​

Get Argo executable version 2.4.2 from Argo Releases on GitHub.

See official Argo documentation.

Test Argo​

Run Hello world workflow to test if Argo has been properly installed. And take a look at the examples provided in Argo documentation to discover how to use the different features available.

argo submit --watch https://raw.githubusercontent.com/argoproj/argo/master/examples/hello-world.yaml
Logged in

You will need to have the oc client installed and be logged in with oc login, see the install documentation page.

Install Argo in your project​

Argo workflows with Helm​

Deploy the Argo Helm chart.

  1. Install and use helm
  2. Add the Helm charts repository:
helm repo add argo https://argoproj.github.io/argo-helm
  1. Install chart:
helm install my-argo argo/argo --version 0.15.2

ArgoCD Operator​

Ask on the DSRI Slack #helpdesk channel to have the ArgoCD Operator installed in your project.

Uninstall argo​

On Ubuntu​

sudo rm /usr/local/bin/argo

You can now reinstall a newer version of Argo.

Run workflows to convert structured data to RDF​

We will use examples from the MaastrichtU-IDS/d2s-core project.

Clone the repository​

git clone --recursive https://github.com/MaastrichtU-IDS/d2s-project-template.git
cd d2s-project-template

Authenticate to the OpenShift cluster using oc login .

Workflow to convert XML files to RDF​

argo submit d2s-core/argo/workflows/d2s-workflow-transform-xml.yaml \
  -f support/config/config-transform-xml-drugbank.yml
Provide config files

Config files can be provided using the -f arguments, but are not necessary.

argo submit d2s-core/argo/workflows/d2s-workflow-transform-xml-dag.yaml \
  -f support/config/config-transform-xml-drugbank.yml

Workflow to convert CSV files to RDF​

  • Steps-based workflow for CSV files
argo submit d2s-core/argo/workflows/d2s-workflow-transform-csv.yaml \
  -f support/config/config-transform-csv-stitch.yml
  • DAG workflow for CSV files
argo submit d2s-core/argo/workflows/d2s-workflow-transform-csv-dag.yaml \
  -f support/config/config-transform-csv-stitch.yml
Solve issue

Try this to solve issue related to steps services IP: {{steps.nginx-server.pod-ip}}

Argo commands​

List running Argo workflows​

argo list

Stop a workflow​

argo terminate my-workflow
Workflow

This might not stop the workflow, in this case use:

argo delete my-workflow

Delete a workflow​

argo delete my-workflow

Debug a workflow​

Get into a container, to understand why it bugs, by creating a YAML with the command tail -f /dev/null to keep it hanging.

See the example in the d2s-argo-workflow repository:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: test-devnull-argo-
spec:
  entrypoint: execute-workflow

  # Use existing volume
  volumes:
  - name: workdir
    persistentVolumeClaim:
      claimName: pvc-mapr-projects-test-vincent 

  templates:
  - name: execute-workflow
    steps:
    - - name: run-rdfunit
        template: rdfunit
   
  - name: rdfunit
    container:
      image: umids/rdfunit:latest
      command: [tail]
      args: ["-f", "/dev/null"]
      volumeMounts:
      - name: workdir
        mountPath: /data
        subPath: dqa-workspace

Then start the workflow:

argo submit --serviceaccount argo tests/test-devnull-argo.yaml

And connect with the Shell (change the pod ID to your pod ID):

oc rsh test-devnull-argo-pod