I have been working on an enhanced python version of the benchmark script in Linux. I have included the capability to read all apps with args from the BenchCFG file and run them on all cores or a maximum number of cores specified on the command line. Still under development. I was hoping for some help on the following items:

1) I remember using a GPU command line option a long time ago to specify which device to use. I think it was -Device x (where x is a number from 0 to Num of GPUs - 1). The experience was in Windows, but when I try this in Linux, I get a segmentation fault. Is this the correct way to specify a device with the Linux app?

2) I would like to include the current stock apps as reference apps. Is there a location to download these?

3) I would like to include a command line option to run test WUs that simulate specific signals. Is there a standard test set of WUs for this purpose?

4) I would like to indicate progress made while the benchmark is running, similar to the current boinc manager. Does anyone know the mechanism used?

Thanks!

---

GitHub: Ricks-Lab
Instagram: ricks_labs

I'm sorry that I can't be of any help to your questions. But I appreciate you making an attempt to make a useful too.

Just out of curiosity, once you finish the MB benchmark tool, do you think you'd be interested in creating a new AP benchmark tool? the current one is quite clunky and we could use a more polished tool.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

I have been working on an enhanced python version of the benchmark script in Linux. I have included the capability to read all apps with args from the BenchCFG file and run them on all cores or a maximum number of cores specified on the command line. Still under development. I was hoping for some help on the following items:

1) I remember using a GPU command line option a long time ago to specify which device to use. I think it was -Device x (where x is a number from 0 to Num of GPUs - 1). The experience was in Windows, but when I try this in Linux, I get a segmentation fault. Is this the correct way to specify a device with the Linux app?

2) I would like to include the current stock apps as reference apps. Is there a location to download these?

3) I would like to include a command line option to run test WUs that simulate specific signals. Is there a standard test set of WUs for this purpose?

4) I would like to indicate progress made while the benchmark is running, similar to the current boinc manager. Does anyone know the mechanism used?

Thanks!

Unfortunately I only know of the /download directory for the BOINC applications. Nothing for the science apps. You could set up a dummy test machine for getting the stock science apps and then copy over the executables.

There are quite a few test WU's for MB V8 over at Lunatics in the Test and Benchmark Tools directory. You can always use the current tasks in your cache for test WU's also. If you use the work unit download generator tools you can retrieve the finished tasks that pique your interest before they are purged from the database.

Richard or Jord might know the mechanism the Client uses to calculate the progress indicator.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

I'm sorry that I can't be of any help to your questions. But I appreciate you making an attempt to make a useful too.

Just out of curiosity, once you finish the MB benchmark tool, do you think you'd be interested in creating a new AP benchmark tool? the current one is quite clunky and we could use a more polished tool.

I am running with AMD GPUs which give invalid Astropulse results when using drivers newer than 15.12, so I am unable to run Astropulse. So I am unlikely to take this on. Not sure if AP will have new life with any of the new data sources.

---

GitHub: Ricks-Lab
Instagram: ricks_labs

I have encountered a new problem. When I try to pause BOINC with the following subprocess call:

cmd = subprocess.Popen("/home/boinc/BOINC/boinccmd --set_run_mode never 172800", shell=True, stdout=subprocess.PIPE)

I get the error:

Operation failed: authentication error

BOINC is running as me and I am executing the python script as me. Any ideas?

Maybe this page for python will help. https://stackoverflow.com/questions/4106565/newbie-python-subprocess-write-error-broken-pipe

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

Maybe this page for python will help. https://stackoverflow.com/questions/4106565/newbie-python-subprocess-write-error-broken-pipe

Since I am actually getting output from the command (authentication error), I don't think it is a broken pipe. Plus I have successfully used subprocess many other cases in the program. I am going to try setting the setuid bit on boinccmd to see if that makes a difference.

Maybe this page for python will help. https://stackoverflow.com/questions/4106565/newbie-python-subprocess-write-error-broken-pipe

Since I am actually getting output from the command (authentication error), I don't think it is a broken pipe. Plus I have successfully used subprocess many other cases in the program. I am going to try setting the setuid bit on boinccmd to see if that makes a difference.

This one is now solved. I had to be in the boinc home directory when calling boinccmd.

Maybe this page for python will help. https://stackoverflow.com/questions/4106565/newbie-python-subprocess-write-error-broken-pipe

Since I am actually getting output from the command (authentication error), I don't think it is a broken pipe. Plus I have successfully used subprocess many other cases in the program. I am going to try setting the setuid bit on boinccmd to see if that makes a difference.

This one is now solved. I had to be in the boinc home directory when calling boinccmd.

or you need to specify the password file if you don't want to chdir to boinc home directory :
/<boinc_path>/boinccmd --host <hostname>:<boinc_port> --passwd <gui_rpc_auth.cfg value> --set_gpu_mode <mode_value>

:<boinc_port> is optional if you use the standard port (31416)

---

I have been working on an enhanced python version of the benchmark script in Linux. I have included the capability to read all apps with args from the BenchCFG file and run them on all cores or a maximum number of cores specified on the command line. Still under development. I was hoping for some help on the following items:

1) I remember using a GPU command line option a long time ago to specify which device to use. I think it was -Device x (where x is a number from 0 to Num of GPUs - 1). The experience was in Windows, but when I try this in Linux, I get a segmentation fault. Is this the correct way to specify a device with the Linux app?

To get the list of available card :
nvidia-smi --query-gpu=index,name --format=csv,noheader
To specify the first card :
nvidia-smi --query-gpu=name --format=csv,noheader -i 0

---

I have been working on an enhanced python version of the benchmark script in Linux. I have included the capability to read all apps with args from the BenchCFG file and run them on all cores or a maximum number of cores specified on the command line. Still under development. I was hoping for some help on the following items:

1) I remember using a GPU command line option a long time ago to specify which device to use. I think it was -Device x (where x is a number from 0 to Num of GPUs - 1). The experience was in Windows, but when I try this in Linux, I get a segmentation fault. Is this the correct way to specify a device with the Linux app?

To get the list of available card :
nvidia-smi --query-gpu=index,name --format=csv,noheader
To specify the first card :
nvidia-smi --query-gpu=name --format=csv,noheader -i 0

I am actually looking for the seti MB app command line option, so when I execute the job, I can specify which gpu to use. Is the mechnism for this different than on windows where it seems to be a command line option?

Maybe this page for python will help. https://stackoverflow.com/questions/4106565/newbie-python-subprocess-write-error-broken-pipe

Since I am actually getting output from the command (authentication error), I don't think it is a broken pipe. Plus I have successfully used subprocess many other cases in the program. I am going to try setting the setuid bit on boinccmd to see if that makes a difference.

This one is now solved. I had to be in the boinc home directory when calling boinccmd.

or you need to specify the password file if you don't want to chdir to boinc home directory :
/<boinc_path>/boinccmd --host <hostname>:<boinc_port> --passwd <gui_rpc_auth.cfg value> --set_gpu_mode <mode_value>

:<boinc_port> is optional if you use the standard port (31416)

This is good to know, but since I will eventually make the script available publicly, I think the directory change will be more appropriate.

I have been working on an enhanced python version of the benchmark script in Linux. I have included the capability to read all apps with args from the BenchCFG file and run them on all cores or a maximum number of cores specified on the command line. Still under development. I was hoping for some help on the following items:

1) I remember using a GPU command line option a long time ago to specify which device to use. I think it was -Device x (where x is a number from 0 to Num of GPUs - 1). The experience was in Windows, but when I try this in Linux, I get a segmentation fault. Is this the correct way to specify a device with the Linux app?

To get the list of available card :
nvidia-smi --query-gpu=index,name --format=csv,noheader
To specify the first card :
nvidia-smi --query-gpu=name --format=csv,noheader -i 0

I am actually looking for the seti MB app command line option, so when I execute the job, I can specify which gpu to use. Is the mechnism for this different than on windows where it seems to be a command line option?

this is what ps show on my computer for the GPU :
../../projects/setiathome.berkeley.edu/setiathome_x41zc_x86_64-pc-linux-gnu_cuda92 -nobs -pfb 16 --device 0

for the CPU, I don't know if you can select on which core you want to run (they are all the same inside a CPU).

---

for the CPU, I don't know if you can select on which core you want to run (they are all the same inside a CPU).

If one wanted a specific thread, or mask even, one could use something like:
schedtool -v -a 1,3,5 `pidof setiathome_x41zi_x86_64-pc-linux-gnu_cuda80`

As long as the app name is unique enough, it can be done with GPU or CPU tasks.

for the CPU, I don't know if you can select on which core you want to run (they are all the same inside a CPU).

If one wanted a specific thread, or mask even, one could use something like:
schedtool -v -a 1,3,5 `pidof setiathome_x41zi_x86_64-pc-linux-gnu_cuda80`

As long as the app name is unique enough, it can be done with GPU or CPU tasks.

Thanks Brent! This will be useful in the future. Still need to figure out why I can't specify the GPU to be used for a job...

Since your taking notes for the future, it also does Priority (Nice Levels):
# Assign CPU Priority (19=Nice, -20=High)
schedtool -n -20 `pidof astropulse_7.08_x86_64-pc-linux-gnu__opencl_nvidia_100`

Concerning the starting of GPU tasks on a specified device, I have checked the MB app source code and found that "--device n" is correct, so maybe there is something else wrong with my code. I am using this chunk of code to start both GPU and CPU tasks:

                    #Execute job
                    shutil.copy2(slots.list[slot_num].job.wu_path + slots.list[slot_num].job.wu_name,
                            slots.list[slot_num].slot_dir +"/" + mb_const.activeWU)
                    if plat == "GPU":
                        for file_str in glob.glob(env.gpu_app_path + "*.cl"):
                            shutil.copy2(file_str, slots.list[slot_num].slot_dir +"/")
                    if plat == "CPU":
                        cmd_str_list = [env.cpu_app_path + slots.list[slot_num].job.app_name , slots.list[slot_num].job.app_args]
                    else:
                        device_arg = "--device " + str(slots.list[slot_num].device)
                        cmd_str_list = [env.gpu_app_path + slots.list[slot_num].job.app_name,
                                device_arg + " " + slots.list[slot_num].job.app_args]
                    if mb_const.DEBUG == True: print(cmd_str_list)
                    os.chdir(slots.list[slot_num].slot_dir)
                    slots.list[slot_num].job.cmd = subprocess.Popen(cmd_str_list, shell=False, stdout=subprocess.PIPE)
                    slots.list[slot_num].job.start_time = datetime.utcnow()
                    os.chdir(env.current_dir)

It works fine for CPU tasks, but still not working for GPU. Here is what cmd_str_list looks like for a CPU task:

['/home/rick/PyDev/benchMB/APPS_CPU/MBv8_8.05r3345_avx_linux64', '--nographics'] 

Maybe there is still something I have not prepared properly in the working directory for a GPU job.

@ReuiKe
Re: downloadable stock apps...
I don't remember how I discovered it, but useful for those of us "anonymous platform" hosts that don't get the automatic downloads of the science apps. Even if we only want to run them as benchmarks to compare to optimized apps.
In the following list the application name, as found in the ->Computing ->Applications web page; then the full application file name.

Linux/x86_64    8.00
setiathome_8.00_x86_64-pc-linux-gnu
---
Linux/x86_64    8.22(opencl_nvidia_sah)
setiathome_8.22_x86_64-pc-linux-gnu__opencl_nvidia_sah
---
Linux/x86_64   8.22(opencl_nvidia_SoG)
setiathome_8.22_x86_64-pc-linux-gnu__opencl_nvidia_SoG
---

Point a browser to:
http://boinc2.ssl.berkeley.edu/sah/download_fanout/<file name> #without the < > of course and it will download to you.
These are the only stock apps that I have in hand and could verify this method. But, no doubt, you will see a pattern in the morphing from Applications description to the downloadable file name, in case you want additional stock apps.
(I hope I'm not revealing any Berkeley secrets here; I have not found any on-line index, or "How-To" , for fetching the stock apps.)

I will follow this thread for progress reports as I am frequently motivated to do benchmark experiments of my own.

AZGene;

---

@ReuiKe
Re: downloadable stock apps...
I don't remember how I discovered it, but useful for those of us "anonymous platform" hosts that don't get the automatic downloads of the science apps. Even if we only want to run them as benchmarks to compare to optimized apps.
In the following list the application name, as found in the ->Computing ->Applications web page; then the full application file name.

Linux/x86_64    8.00
setiathome_8.00_x86_64-pc-linux-gnu
---
Linux/x86_64    8.22(opencl_nvidia_sah)
setiathome_8.22_x86_64-pc-linux-gnu__opencl_nvidia_sah
---
Linux/x86_64   8.22(opencl_nvidia_SoG)
setiathome_8.22_x86_64-pc-linux-gnu__opencl_nvidia_SoG
---

Point a browser to:
http://boinc2.ssl.berkeley.edu/sah/download_fanout/<file name> #without the < > of course and it will download to you.
These are the only stock apps that I have in hand and could verify this method. But, no doubt, you will see a pattern in the morphing from Applications description to the downloadable file name, in case you want additional stock apps.
(I hope I'm not revealing any Berkeley secrets here; I have not found any on-line index, or "How-To" , for fetching the stock apps.)

I will follow this thread for progress reports as I am frequently motivated to do benchmark experiments of my own.

AZGene;

Thanks Gene, That worked!

I hope to have a functional beta by the end of next weekend. Things I need to complete:
1) Write meaningful data to testData directory. Will likely leverage the comparison routine used in the previous package
2) Run and store results for reference apps
3) Figure out how to make previous compiled kernels available for new runs. I am using a slots approach, similar to boinc. It looks like boinc writes compiled kernels to a common directory, but in my case, the job running in a slot is expecting the kernels in the slots dir. Still some digging to do on this one.

I have figured out this issue. I had to add a space before the "-device n". Not sure why it is needed since I don't put a space in front of the cpu app argument string.

Concerning the starting of GPU tasks on a specified device, I have checked the MB app source code and found that "--device n" is correct, so maybe there is something else wrong with my code. I am using this chunk of code to start both GPU and CPU tasks:

                    #Execute job
                    shutil.copy2(slots.list[slot_num].job.wu_path + slots.list[slot_num].job.wu_name,
                            slots.list[slot_num].slot_dir +"/" + mb_const.activeWU)
                    if plat == "GPU":
                        for file_str in glob.glob(env.gpu_app_path + "*.cl"):
                            shutil.copy2(file_str, slots.list[slot_num].slot_dir +"/")
                    if plat == "CPU":
                        cmd_str_list = [env.cpu_app_path + slots.list[slot_num].job.app_name , slots.list[slot_num].job.app_args]
                    else:
                        device_arg = "--device " + str(slots.list[slot_num].device)
                        cmd_str_list = [env.gpu_app_path + slots.list[slot_num].job.app_name,
                                device_arg + " " + slots.list[slot_num].job.app_args]
                    if mb_const.DEBUG == True: print(cmd_str_list)
                    os.chdir(slots.list[slot_num].slot_dir)
                    slots.list[slot_num].job.cmd = subprocess.Popen(cmd_str_list, shell=False, stdout=subprocess.PIPE)
                    slots.list[slot_num].job.start_time = datetime.utcnow()
                    os.chdir(env.current_dir)

It works fine for CPU tasks, but still not working for GPU. Here is what cmd_str_list looks like for a CPU task:

['/home/rick/PyDev/benchMB/APPS_CPU/MBv8_8.05r3345_avx_linux64', '--nographics'] 

Maybe there is still something I have not prepared properly in the working directory for a GPU job.