Distributing 4-bit workunits

Message boards : News : Distributing 4-bit workunits
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 6 · Next

AuthorMessage
Profile Eric J Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 15 Mar 05
Posts: 1547
Credit: 26,959,651
RAC: 1,609
United States
Message 58603 - Posted: 15 Jun 2016, 1:24:25 UTC - in response to Message 58599.  


Is it for GBT-only data?


It's for both Arecibo and GBT.
ID: 58603 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,340,341
RAC: 59
United States
Message 58604 - Posted: 15 Jun 2016, 2:54:56 UTC

I'm guessing that since these blc3_2bit_guppi have file sizes of 704KB that they are the 4-bit work units?
ID: 58604 · Report as offensive
Profile Eric J Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 15 Mar 05
Posts: 1547
Credit: 26,959,651
RAC: 1,609
United States
Message 58605 - Posted: 15 Jun 2016, 3:49:24 UTC - in response to Message 58604.  

That's correct.
ID: 58605 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,340,341
RAC: 59
United States
Message 58606 - Posted: 15 Jun 2016, 6:13:27 UTC

I'm seeing a lot of -9 overflows with work units starting with 24mr10

Anyone else seeing that? Or is it just my machine?
ID: 58606 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 15 Jun 16
Posts: 45
Credit: 1,836,741
RAC: 0
Australia
Message 58609 - Posted: 15 Jun 2016, 8:19:14 UTC - in response to Message 58606.  

I'm seeing a lot of -9 overflows with work units starting with 24mr10

Anyone else seeing that? Or is it just my machine?

I only had one of them.
Suspended current work to run it, and it lasted 10 seconds. Looking dodgy.
Grant
Darwin NT.
ID: 58609 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester

Send message
Joined: 11 Nov 12
Posts: 851
Credit: 2,991,411
RAC: 228
United Kingdom
Message 58610 - Posted: 15 Jun 2016, 9:07:09 UTC

Well I have set one of currently unused machines to crunch here.

As they say every little helps!!
ID: 58610 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 15 Jun 16
Posts: 45
Credit: 1,836,741
RAC: 0
Australia
Message 58611 - Posted: 15 Jun 2016, 9:17:06 UTC - in response to Message 58609.  
Last modified: 15 Jun 2016, 9:33:02 UTC

I'm seeing a lot of -9 overflows with work units starting with 24mr10

Anyone else seeing that? Or is it just my machine?

I only had one of them.
Suspended current work to run it, and it lasted 10 seconds. Looking dodgy.

Just got another one, it's crunching normally.

EDIT- That and another crunched OK, another 2 only lasted 10 and 11 seconds.
Grant
Darwin NT.
ID: 58611 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 15 Jun 16
Posts: 45
Credit: 1,836,741
RAC: 0
Australia
Message 58612 - Posted: 15 Jun 2016, 9:18:34 UTC - in response to Message 58610.  

Well I have set one of currently unused machines to crunch here.

As they say every little helps!!

Only another 3,300 WUs to finish off.
:-)
Grant
Darwin NT.
ID: 58612 · Report as offensive
Profile Raistmer
Volunteer tester
Avatar

Send message
Joined: 18 Aug 05
Posts: 2423
Credit: 15,878,738
RAC: 0
Russia
Message 58614 - Posted: 15 Jun 2016, 10:36:44 UTC

Got 4-bits Arecibo tasks:

<receiver_cfg>
<s4_id>11</s4_id>
<name>Arecibo 1.4GHz Array, Beam 4, Pol 0</name>
<beam_width>0.0500000007</beam_width>
<center_freq>1420</center_freq>
<latitude>18.3538056</latitude>
<longitude>-66.7552222</longitude>
<elevation>497</elevation>
<diameter>168</diameter>
<az_orientation>180</az_orientation>
<az_corr_coeff length=99 encoding="x-csv">
-37,-6.05,92.35,-731.21,-1013.97,-24.53,-11.19,9.18,106.04,3.02,-1.74,
-3.46,1.29
</az_corr_coeff>
<zen_corr_coeff length=99 encoding="x-csv">
-57.55,-95.56,-4.13,141.69,677.51,-10.41,-7.71,-10.39,0.08,0.43,-0.62,
0.03,-0.36
</zen_corr_coeff>
<array_az_ellipse>329.06</array_az_ellipse>
<array_za_ellipse>384.005</array_za_ellipse>
<array_angle>-60</array_angle>
<min_vgc>512</min_vgc>
<polarization>linear</polarization>
</receiver_cfg>
<recorder_cfg>
<name>seti_dr2_ALFA</name>
<bits_per_sample>2</bits_per_sample>
<sample_rate>2500000</sample_rate>
<beams>14</beams>
<version>1.99000001</version>
</recorder_cfg>
<splitter_cfg>
<version>0.300000012</version>
<data_type>encoded</data_type>
<fft_len>256</fft_len>
<ifft_len>1</ifft_len>
<filter>polyphase</filter>
<window>hanning</window>
<samples_per_wu>1048576</samples_per_wu>
<highpass>0</highpass>
<blanker_filter>randomize</blanker_filter>
<pfb_ntaps>16</pfb_ntaps>
<pfb_width_factor>1.04999995</pfb_width_factor>
<wu_bits_per_sample>4</wu_bits_per_sample>
</splitter_cfg>
News about SETI opt app releases: https://twitter.com/Raistmer
ID: 58614 · Report as offensive
Profile Raistmer
Volunteer tester
Avatar

Send message
Joined: 18 Aug 05
Posts: 2423
Credit: 15,878,738
RAC: 0
Russia
Message 58616 - Posted: 15 Jun 2016, 10:48:46 UTC - in response to Message 58603.  
Last modified: 15 Jun 2016, 10:49:21 UTC


Is it for GBT-only data?


It's for both Arecibo and GBT.

As I could recall there was some discussion with Josef Segur to enable compression inside workunit. There are even test compressed tasks available in Lunatics's collection (processed OK by known modern MB apps).
Maybe worth to enable such compression now?
News about SETI opt app releases: https://twitter.com/Raistmer
ID: 58616 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 15 Jun 16
Posts: 45
Credit: 1,836,741
RAC: 0
Australia
Message 58617 - Posted: 15 Jun 2016, 10:58:21 UTC - in response to Message 58616.  

Would compression have much effect on file sizes?
From memory truly random data doesn't compress much (if at all), so with WUs the only parts that would compress would be the added reference data.
Grant
Darwin NT.
ID: 58617 · Report as offensive
Richard Haselgrove
Volunteer tester

Send message
Joined: 3 Jan 07
Posts: 1451
Credit: 3,266,428
RAC: 0
United Kingdom
Message 58618 - Posted: 15 Jun 2016, 11:23:40 UTC - in response to Message 58617.  

You can easily test - take a sample datafile from your cache and run it through gz or zip compression tools (I doubt the project would go as far as LZMA, so not 7-zip for this test).

2-bit MB tasks shrink fron ~357 KB to ~267 KB, or 25%, because of the 64 character ASCII encoding. AP data is sent in a denser binary encoding, which barely compresses at all (except when the B3 P0 bit is stuck...)

Now that our download channel seems to have enough bandwidth, would the server processing overhead of compression be worth a 25% saving?
ID: 58618 · Report as offensive
Profile Eric J Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 15 Mar 05
Posts: 1547
Credit: 26,959,651
RAC: 1,609
United States
Message 58620 - Posted: 15 Jun 2016, 15:13:40 UTC - in response to Message 58616.  

We could just send binary data rather than encoded data to get most of the same benefit without loading the splitters or servers. Maybe I'll try that next.
ID: 58620 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,340,341
RAC: 59
United States
Message 58621 - Posted: 15 Jun 2016, 15:40:29 UTC - in response to Message 58611.  
Last modified: 15 Jun 2016, 15:43:54 UTC


I only had one of them.
Suspended current work to run it, and it lasted 10 seconds. Looking dodgy.

Just got another one, it's crunching normally.

EDIT- That and another crunched OK, another 2 only lasted 10 and 11 seconds.


I checked your "shorts" as well and they have the same results

SETI@Home Informational message -9 result_overflow


So seems there is a batch shorties in there

Edit... SoG nonmodified 13 minutes, cuda 42 - 33 minutes, sah in the bull pen and waiting to start
ID: 58621 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,340,341
RAC: 59
United States
Message 58622 - Posted: 15 Jun 2016, 17:03:20 UTC - in response to Message 58621.  

Picked up 5 errors on the SaH app

ERROR: Possible wrong computation state on GPU, host needs reboot or maintenance
GPU device sync requested... ...GPU device synche


https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065993
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065968
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24066051
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065976
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065955

GPUs run fine for cuda SoG and Cuda 42. Not sure why these were erroring out
ID: 58622 · Report as offensive
Profile Raistmer
Volunteer tester
Avatar

Send message
Joined: 18 Aug 05
Posts: 2423
Credit: 15,878,738
RAC: 0
Russia
Message 58623 - Posted: 15 Jun 2016, 18:21:29 UTC - in response to Message 58622.  

Picked up 5 errors on the SaH app

ERROR: Possible wrong computation state on GPU, host needs reboot or maintenance
GPU device sync requested... ...GPU device synche


https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065993
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065968
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24066051
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065976
https://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24065955

GPUs run fine for cuda SoG and Cuda 42. Not sure why these were erroring out


Too big autocorr value happened too often. That triggered sanity check.
I discussed this with Eric - decision was leave it as is for now.
News about SETI opt app releases: https://twitter.com/Raistmer
ID: 58623 · Report as offensive
.
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1659
Credit: 12,522,565
RAC: 5,466
Sweden
Message 58625 - Posted: 15 Jun 2016, 19:40:25 UTC
Last modified: 15 Jun 2016, 19:43:56 UTC

Hmm, interesting. With these new 4-bit WU's, it seems as if opencl_nvidia_sah, is coming back into play again. Up to now, opencl_nvidia_SoG has been chosen as the clearly faster app. I haven't been getting any opencl_nvidia_sah tasks for several days, but now after the shift to 4-bit WU's, I'm getting them again.

Now though, it seems as if opencl_nvidia_sah will be competing with SoG with both Arecibo and Guppi tasks. At least that's what I see now initially. With the Guppi tasks, opencl_nvidia_sah seems to be almost 2 minutes faster, but nothing is conclusive yet. Could have been some freak stuff happening.
ID: 58625 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,340,341
RAC: 59
United States
Message 58626 - Posted: 15 Jun 2016, 20:17:48 UTC - in response to Message 58625.  

For me SaH caused some Error while computing. Never saw those with the SoG.

I need to test Raistmer newest r3472 so I'll be back tomorrow and see if I can get some more SoG here.

Like you, the server seems to think SaH is better but I know looking at the times, SoG was better.

I think all those errors while downloading threw a confused the server into thinking SaH was better since only 5 with it vs 50 (download errors)
ID: 58626 · Report as offensive
.
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1659
Credit: 12,522,565
RAC: 5,466
Sweden
Message 58627 - Posted: 15 Jun 2016, 20:23:30 UTC - in response to Message 58626.  
Last modified: 15 Jun 2016, 20:23:56 UTC

For me SaH caused some Error while computing. Never saw those with the SoG.

I need to test Raistmer newest r3472 so I'll be back tomorrow and see if I can get some more SoG here.

Like you, the server seems to think SaH is better but I know looking at the times, SoG was better.

I think all those errors while downloading threw a confused the server into thinking SaH was better since only 5 with it vs 50 (download errors)

Well, I don't know really. It seems to go up and down here, which one is faster. Same AR can at times be faster with opencl_nvidia_SoG, other times, same AR is faster with opencl_nvidia_sah. That goes for both Arecibo and Guppi tasks.

I think I'm getting a tiny bit confused here.

Ah well, at 60, I guess that's just to be expected, and it's just going to get worse over time :-)
ID: 58627 · Report as offensive
.
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1659
Credit: 12,522,565
RAC: 5,466
Sweden
Message 58628 - Posted: 16 Jun 2016, 1:55:36 UTC
Last modified: 16 Jun 2016, 2:08:38 UTC

Stock CPU app, also seems to work just fine with these 4-bit workunits:

http://setiweb.ssl.berkeley.edu/beta/result.php?resultid=24070149

Although stock CPU app, is painfully slow, compared to MB8_win_x64_AVX_VS2010_r3330.exe. Stock CPU app, takes more than twice the time, almost 2.5 times slower.

Edit: But a lot of that slowness, is due to me now running the iGPU with one task too, something I don't do on main with the opt CPU app. That slows down CPU apps so much that it's almost crazy.
ID: 58628 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 . . . 6 · Next

Message boards : News : Distributing 4-bit workunits


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.