Distributing 4-bit workunits

Message boards : News : Distributing 4-bit workunits
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 6 · Next

AuthorMessage
Profile Eric J Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 15 Mar 05
Posts: 1542
Credit: 25,792,165
RAC: 7,755
Message 58569 - Posted: 13 Jun 2016, 20:45:57 UTC

This has been in the works for a while. I'll be distributing some workunits with 4-bit complex samples. They'll be twice the size of the current 2-bit workunits, but will give us 46% reduction in noise power.

It's possible that some of the app versions date from before the 4-bit quantization code existed in the client (but I don't think so). If things are going to break it should be obvious pretty quickly.

The change should occur this afternoon.
ID: 58569 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58570 - Posted: 13 Jun 2016, 21:16:11 UTC - in response to Message 58569.  
Last modified: 13 Jun 2016, 21:17:44 UTC

This has been in the works for a while. I'll be distributing some workunits with 4-bit complex samples. They'll be twice the size of the current 2-bit workunits, but will give us 46% reduction in noise power.

It's possible that some of the app versions date from before the 4-bit quantization code existed in the client (but I don't think so). If things are going to break it should be obvious pretty quickly.

The change should occur this afternoon.

Oh my, oh my, and I'm running Beta 100% now, since a couple of day. SoG has been chosen by the system, and I also run the iGPU app.

Both with optimized settings. I guess I will soon see, if my computer will start emitting blue smoke :-)
ID: 58570 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58571 - Posted: 13 Jun 2016, 21:24:30 UTC - in response to Message 58570.  

Eric should we be running stock for these work units or will the SoG work?
ID: 58571 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58572 - Posted: 13 Jun 2016, 21:38:54 UTC
Last modified: 13 Jun 2016, 21:40:04 UTC

I'm running stock, and the system has decided that SoG is the fastest, so I only get SoG for my GTX980. Also my iGPU runs stock of course.

I wonder if these 4-bit workunits will also take twice as long, since they will be twice the size of the current 2-bit workunits?
ID: 58572 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58573 - Posted: 13 Jun 2016, 21:46:26 UTC - in response to Message 58572.  

Yea, I just turned my quad over here as well. It's working it's way thru the different apps to see which is fastest.

But if they work with SoG then I have to wonder if we put in the commandlines, can we speed up the complete time? Of course I will run them stock for a while, once I get 100 or so then I can start to play around with the commandlines.
ID: 58573 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58574 - Posted: 13 Jun 2016, 21:54:43 UTC - in response to Message 58573.  
Last modified: 13 Jun 2016, 21:57:47 UTC

Yea, I just turned my quad over here as well. It's working it's way thru the different apps to see which is fastest.

But if they work with SoG then I have to wonder if we put in the commandlines, can we speed up the complete time? Of course I will run them stock for a while, once I get 100 or so then I can start to play around with the commandlines.

Well, I run stock as I said (it's stock even if you use commandline options, Running "Anonymous Platform", is not stock though), but with commandline options set. Both for SoG and for iGPU.

My SoG commandline is:
-cpu_lock -sbs 512 -period_iterations_num 2 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64 -instances_per_device 3 -total_GPU_instances_num 4

And My iGPU commandline is:
-cpu_lock -sbs 256 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 512 -instances_per_device 1 -total_GPU_instances_num 4
ID: 58574 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58575 - Posted: 13 Jun 2016, 22:07:45 UTC - in response to Message 58574.  

I'm just going to be doing 1 at a time first. Need a baseline with the new data
ID: 58575 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58576 - Posted: 13 Jun 2016, 22:12:14 UTC - in response to Message 58575.  

I'm just going to be doing 1 at a time first. Need a baseline with the new data

OK, we'll see what happens. I'm fully prepared for blue smoke here :-)
ID: 58576 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58577 - Posted: 13 Jun 2016, 22:37:53 UTC - in response to Message 58576.  

One thing I keep forgetting to ask, how will we know if these are the 4 bit work units?

Is there a prefix in the name that will tell us?
ID: 58577 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58578 - Posted: 13 Jun 2016, 22:47:26 UTC - in response to Message 58577.  
Last modified: 13 Jun 2016, 22:49:36 UTC

One thing I keep forgetting to ask, how will we know if these are the 4 bit work units?

Is there a prefix in the name that will tell us?

Example of the ones we've been getting up to now:

blc3_2bit_guppi_57451_20612_HIP62472_0007.15822.831.17.20.54.vlar_1

The bold part is what you should look at. That should read 4bit_guppi, with the new ones I guess.
ID: 58578 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58579 - Posted: 13 Jun 2016, 22:50:52 UTC - in response to Message 58578.  

ah, ok thanks
ID: 58579 · Report as offensive
Profile Eric J Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 15 Mar 05
Posts: 1542
Credit: 25,792,165
RAC: 7,755
Message 58580 - Posted: 13 Jun 2016, 23:31:32 UTC

Actually, it's going to take a bit longer than I thought for them to get distributed. I thought that the ready to send count was small, but in reality the status page was wrong (there were 8000).

There won't be any easy way to tell what you've got apart from the file size. 2-bit WUs are about 360K. 4-bit WUs are about 620K. The "2bit" or "8bit" designation in the file name is about the input data that the splitter operated upon. (And it's wrong anyway. The 2bit input data is actually 4 bits per complex sample.)

In the workunit file itself, there's a relatively new entry called wu_bits_per_sample which should be 2 past data, and in the new data should say 4.
ID: 58580 · Report as offensive
Profile Eric J Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 15 Mar 05
Posts: 1542
Credit: 25,792,165
RAC: 7,755
Message 58581 - Posted: 13 Jun 2016, 23:37:40 UTC - in response to Message 58580.  

As far as computing, there shouldn't be a significant difference. The old data got turned into floating point numbers (either -1.0 or 1.0) before being processed. The new data gets turned into the same number of floating point values, but now the values are -3.3358750, -1.0, 1.0, or 3.3358750.

The most likely problem you'll encounter is that some version might be compiled with old workunit reading code which will see the file size is wrong and abort immediately.
ID: 58581 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58582 - Posted: 13 Jun 2016, 23:48:23 UTC - in response to Message 58580.  
Last modified: 13 Jun 2016, 23:49:38 UTC

The "2bit" or "8bit" designation in the file name is about the input data that the splitter operated upon. (And it's wrong anyway. The 2bit input data is actually 4 bits per complex sample.)

In the workunit file itself, there's a relatively new entry called wu_bits_per_sample which should be 2 past data, and in the new data should say 4.

Hehe, oh you scientists are complicated :-) And here I thought that the "bit" part in the file name was what it seemed to be.

Geeze, I'm getting old, and tired, very very old and tired :-)
ID: 58582 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58583 - Posted: 14 Jun 2016, 0:23:34 UTC - in response to Message 58580.  
Last modified: 14 Jun 2016, 0:24:32 UTC

but in reality the status page was wrong (there were 8000).


Hmm.. Going to take us a while to burn thru those work units so we can get to the new ones.

depending on how many people are actually over here on Beta.
ID: 58583 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58590 - Posted: 14 Jun 2016, 7:31:47 UTC - in response to Message 58583.  

but in reality the status page was wrong (there were 8000).


Hmm.. Going to take us a while to burn thru those work units so we can get to the new ones.

depending on how many people are actually over here on Beta.

It's you and me, and according to the SSP, 387 other users. :-)
http://setiweb.ssl.berkeley.edu/beta/server_status.php
ID: 58590 · Report as offensive
Profile Raistmer
Volunteer tester
Avatar

Send message
Joined: 18 Aug 05
Posts: 2389
Credit: 15,878,679
RAC: 5
Message 58599 - Posted: 14 Jun 2016, 23:20:54 UTC - in response to Message 58569.  

This has been in the works for a while. I'll be distributing some workunits with 4-bit complex samples. They'll be twice the size of the current 2-bit workunits, but will give us 46% reduction in noise power.

It's possible that some of the app versions date from before the 4-bit quantization code existed in the client (but I don't think so). If things are going to break it should be obvious pretty quickly.

The change should occur this afternoon.


Is it for GBT-only data?
News about SETI opt app releases: https://twitter.com/Raistmer
ID: 58599 · Report as offensive
Tutankhamon (Nothing to test)
Volunteer tester
Avatar

Send message
Joined: 10 Mar 12
Posts: 1308
Credit: 6,039,651
RAC: 49
Message 58600 - Posted: 14 Jun 2016, 23:49:24 UTC
Last modified: 15 Jun 2016, 0:26:23 UTC

First one finished:
http://setiweb.ssl.berkeley.edu/beta/result.php?result_name=blc3_2bit_guppi_57451_20612_HIP62472_0007.9291.0.17.20.62.vlar_0
Looks OK to me.

Edit: and the second one: http://setiweb.ssl.berkeley.edu/beta/result.php?result_name=blc3_2bit_guppi_57451_20612_HIP62472_0007.9291.0.17.20.87.vlar_2

iGPU running one now also. Seems OK so far.

Seems just fine. I got more of them running just now, but I'll go back to FIFO now, finishing the old ones and the SaH WU's.

Edit: And here's the first iGPU 4-bit task finished: http://setiweb.ssl.berkeley.edu/beta/result.php?result_name=blc3_2bit_guppi_57451_20612_HIP62472_0007.9291.416.17.20.16.vlar_0
ID: 58600 · Report as offensive
Wembley
Volunteer tester
Avatar

Send message
Joined: 13 Nov 09
Posts: 12
Credit: 135,674
RAC: 19
Message 58601 - Posted: 15 Jun 2016, 0:22:05 UTC

I turned beta back on and got a bunch, a couple have already finished, but none of the new tasks are showing up on my task list? Is the server just slow or is it borked?
ID: 58601 · Report as offensive
Zalster
Volunteer tester

Send message
Joined: 30 Dec 13
Posts: 258
Credit: 12,035,324
RAC: 0
Message 58602 - Posted: 15 Jun 2016, 0:28:29 UTC - in response to Message 58601.  

Not unusual for the information to be wrong following the normal tuesday downage for maintenance.

Should eventually correct itself in a few hours
ID: 58602 · Report as offensive
1 · 2 · 3 · 4 . . . 6 · Next

Message boards : News : Distributing 4-bit workunits


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.