Skip to content

Commit 0c0d119

Browse files
amila-desilvaamila-desilva
committed
added bash_run.rst and examples file back
1 parent d38e314 commit 0c0d119

File tree

2 files changed

+166
-0
lines changed

2 files changed

+166
-0
lines changed

‎docs/source/bash_run.rst

+115
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
.. _bash_run:
2+
3+
Going deeper with WorkUnitSubmit ``bash_run``
4+
==================================================
5+
6+
Let's start with the ``bash_run`` function definition.
7+
8+
9+
.. py:function:: WorkUnitSubmit.bash_run(self, compiled_file, [options: dict = None, show_command=False])
10+
11+
Compile your workunit with the eclcc compiler
12+
13+
:params: str: compiled_file
14+
:params: dict: options
15+
:return: parsed_response
16+
:rtype: dict
17+
18+
19+
Let's run through a sample example.
20+
21+
.. code-block:: python
22+
:linenos:
23+
24+
import os
25+
26+
from pyhpcc.models.auth import Auth
27+
from pyhpcc.models.hpcc import HPCC
28+
from pyhpcc.models.workunit_submit import WorkunitSubmit as ws
29+
30+
# Example to show how to compile and run an inline ECL query
31+
32+
# Configurations
33+
environment = (
34+
"university.us-hpccsystems-dev.azure.lnrsg.io" # Eg: myuniversity.hpccsystems.io
35+
)
36+
port = "8010" # Eg: 8010
37+
user_name = "user_name" # HPCC username
38+
password = "password" # HPCC password
39+
protocol = "http" # Specify HTTP or HTTPS
40+
cluster = "thor" # Specify the cluster name to be used
41+
ecl_query = """OUTPUT('HELLO WORLD!');""" # ECL Query to execute
42+
job_name = "Basic job submission"
43+
working_folder = os.getcwd() # Folder to generate .ecl, .eclxml, .eclxml.xml
44+
45+
try:
46+
auth_object = Auth(
47+
environment,
48+
port,
49+
user_name,
50+
password,
51+
require_auth=True,
52+
protocol=protocol,
53+
)
54+
hpcc_object = HPCC(auth=auth_object)
55+
work_s = ws(hpcc_object, (cluster,), remove_temp_files=True)
56+
# Create a file for the ECL query we want to execute
57+
file_name = work_s.create_file_name(
58+
query_text=ecl_query,
59+
working_folder=working_folder,
60+
job_name=job_name,
61+
)
62+
# Compile the ECL file
63+
output, output_file = work_s.bash_compile(file_name)
64+
# check if the query is compiled without any errors
65+
if output["status"] == "success":
66+
# Run the ECL file that's compiled
67+
output = work_s.bash_run(output_file, show_command=True)
68+
# Check if workunit is created and wait on it for completion
69+
if (wuid := output["wu_info"]["wuid"]) is not None:
70+
resp = work_s.wu_wait_complete(wuid)
71+
print(resp.json())
72+
73+
except Exception as e:
74+
print(e)
75+
76+
Output of the code above
77+
78+
.. code-block:: bash
79+
80+
{'WUWaitResponse': {'StateID': 3}}
81+
82+
Let's look at what `bash_run` is doing under the hood.
83+
84+
* Set the authorization parameters that are required to run the ecl query on the cluster.
85+
* If no cluster is specified in the ``run config`` options, it finds the least active cluster from the least of clusters provided during ``WorkunitSubmit`` constructor initialization.
86+
* If limit option (``--limit``) isn't provided, it sets the result to default limit ``100``
87+
88+
How to set ``RunConfig`` options and usage of ``show_command`` option:
89+
------------------------------------
90+
91+
Previously we discussed about the defualt result limit (``100``).
92+
What if we also need to set the ssl flag (``-ssl``)
93+
To set the above options, let's see how to do it.
94+
.. code-block:: python
95+
:linenos:
96+
97+
output = work_s.bash_run(output_file, options={"--limit": 200, "-ssl": bool}, show_command=True)
98+
print(output)
99+
100+
* Using the options parameter in bash_run we are able to set the ``limit`` and ``ssl`` flag.
101+
102+
Output of the above code:
103+
104+
.. code-block:: bash
105+
106+
{
107+
"error": {"message": []},
108+
"raw_output": "<Raw output of ecl run>",
109+
"wu_info": {"wuid": "<wuid>", "state": "<state>"},
110+
"misc_info": {"message": []},
111+
"command": "ecl run /path/to/ecl-output/Basic_job_submission.eclxml --limit 200 -ssl --target thor --job-name Basic_job_submission -s university.us-hpccsystems-dev.azure.lnrsg.io --port 8010 -u username -pw ***** -v",
112+
}
113+
114+
* When we set the ``show_command`` in ``bash_run``, the output will include ``ecl run`` command used to execute. See the ``command`` field in the output above.
115+
* **NOTE**: To protect the privacy of the credentials used, the password is masked to `*****`

‎examples/work_unit_hello_world.py

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
import os
2+
3+
from pyhpcc.models.auth import Auth
4+
from pyhpcc.models.hpcc import HPCC
5+
from pyhpcc.models.workunit_submit import WorkunitSubmit as ws
6+
7+
# Example to show how to compile and run an inline ECL query
8+
9+
# Configurations
10+
environment = (
11+
"university.us-hpccsystems-dev.azure.lnrsg.io" # Eg: myuniversity.hpccsystems.io
12+
)
13+
port = "8010" # Eg: 8010
14+
user_name = "user_name" # HPCC username
15+
password = "password" # HPCC password
16+
protocol = "http" # Specify HTTP or HTTPS
17+
cluster = "thor" # Specify the cluster name to be used
18+
ecl_query = """OUTPUT('HELLO WORLD!');""" # ECL Query to execute
19+
job_name = "Basic job submission"
20+
working_folder = os.getcwd() # Folder to generate .ecl, .eclxml, .eclxml.xml
21+
22+
try:
23+
auth_object = Auth(
24+
environment,
25+
port,
26+
user_name,
27+
password,
28+
require_auth=True,
29+
protocol=protocol,
30+
)
31+
hpcc_object = HPCC(auth=auth_object)
32+
work_s = ws(hpcc_object, (cluster,), remove_temp_files=True)
33+
# Create a file for the ECL query we want to execute
34+
file_name = work_s.create_file_name(
35+
query_text=ecl_query,
36+
working_folder=working_folder,
37+
job_name=job_name,
38+
)
39+
# Compile the ECL file
40+
output, output_file = work_s.bash_compile(file_name)
41+
# check if the query is compiled without any errors
42+
if output["status"] == "success":
43+
# Run the ECL file that's compiled
44+
output = work_s.bash_run(output_file)
45+
# Check if workunit is created and wait on it for completion
46+
if (wuid := output["wu_info"]["wuid"]) is not None:
47+
resp = work_s.wu_wait_complete(wuid)
48+
print(resp.json())
49+
50+
except Exception as e:
51+
print(e)

0 commit comments

Comments
 (0)