Detailed Look at an ETL Conductor Workflow | Orkes Conductor Documentation

{
“gh_account”: “netflix”,
“gh_repo”: “conductor”,
“star_offset”: 3800,
“gh_token”: “<github_api_key>”,
“orbit_workspace”: “oss-stats”,
“activity_name”: “starredConductor”,
“orbit_apikey”: “<orbit_api_key>”
}

{
"name": "calculate_start_cutoff",
"taskReferenceName": "calculate_start_cutoff_ref",
"inputParameters": {
"evaluatorType": "javascript",
"expression": "new Date(Date.now() - 86400 * 1000).toISOString();"
},
"type": "INLINE"
}
"inputParameters": {
"offset": "${workflow.input.star_offset}",
"stargazers": "${Get_repo_details_ref.output.response.body.stargazers_count}"

},
"loopCondition": "if ($.get_all_stars_loop_ref['iteration'] < Math.ceil(($.stargazers-$.offset)/100)) { true; } else { false; }",

{
“name”: “100_stargazers”,
“taskReferenceName”: “hundred_stargazers_ref”,
“inputParameters”: {
“counter”: “${pagination_calc_ref.output.result}”,
“http_request”: {
“uri”: “https://api.github.com/repos/${workflow.input.gh_account}/${workflow.input.gh_repo}/stargazers?page=${pagination_calc_ref.output.result}&per_page=100",
“method”: “GET”,
“headers”: {
“Authorization”: “token ${workflow.input.gh_token}”,
“Accept”: “application/vnd.github.v3.star+json”
}
}
},
“type”: “HTTP”
},

--

--

Developer Relations. Web and Native Mobile Performance engineer. Passionate about making the web faster. Available for short/long term consultation.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store