Any rough guess how much hardware it would take to crunch real time? If we assume WUs are uploaded directly and ignore any of the backend issues, how many let's say Nvidia 1080s would it take?

Let's also assume all of these 1080s would scale linearly to make it easy, and some normalizing of per year data coming in, averaging of WU types.

Lots of assumptions, but would that number be 10k, 100k, 1M, some really rough calc or guess would still be an interesting conversation.

---

Thousands!
Each task we run is a very narrow band sample of two seconds worth of data from one of the receiver channels and that takes at least a couple of minutes - so we are looking for a leverage of at least 100:1 in terms of performance.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

To do that kind of scaling (thousands) we are either talking about one of those "super" computers I keep reading about or we are talking about a LOT of boxes. Something like W3Perl runs https://setiathome.berkeley.edu/show_user.php?userid=407.

Tom

---

A proud member of the OFA (Old Farts Association).

It is an interesting thought problem. We don't get continuous data. We won't have any data from Aricebo for the next 3 days. The blc data from greenbank seems to take us a week to just process one section of a day's data. I don't know how often we get a day of data though. The files we are processing now (58406) is Nov 15, 2018 , so we aren't processing very old data. The current status shows files from that one day of blc4, blc5, blc6, 12, 14, 15, 16. I can't remember if there were more that have finished already. That is 7 weeks roughly to process one day of data from Greenbank. We have yet to see Parkes data, so I have no idea how that factors into the equation. The letter that went out in December did say they needed more people to join in the Seti effort though.

The files we are processing now (58406) is Nov 15, 2018 , so we aren't processing very old data.

That is interesting, if we are really that far in a processing deficit, one would think we'd be way behind, years even, or the data is not loaded chronological order.

---

To do that kind of scaling (thousands) we are either talking about one of those "super" computers I keep reading about or we are talking about a LOT of boxes. Something like W3Perl runs https://setiathome.berkeley.edu/show_user.php?userid=407.

Tom

Only hundred...not thousand ;)

---

Thousands!
Each task we run is a very narrow band sample of two seconds worth of data from one of the receiver channels and that takes at least a couple of minutes - so we are looking for a leverage of at least 100:1 in terms of performance.

According to the "About SETI@home tab", aren't workunits made up of 107 second tasks, that overlap slightly?

Here's a quote:
SETI@home looks at 2.5 MHz of data, centered at 1420 MHz. This is still too broad a spectrum to send to you for analysis, so we break this spectrum space up into 256 pieces, each 10 kHz wide (more like 9766 Hz, but we'll simplify the numbers to make calculations easier to see). This is done with a software program called the "splitter". These 10 kHz pieces are now more manageable in size. To record signals up to 10 KHz you have to record the bits at 20,000 bits per second (kbps). (This is called the Nyquist frequency.) We send you about 107 seconds of this 10 kHz (20kbps) data. 100 seconds times 20,000 bits equals 2,000,000 bits, or about 0.25 megabyte given that there are 8 bits per byte. Again, we call this 0.25 megabyte chunk a "work-unit." We also send you lots of additional info about the work-unit, so the total comes out to about 340 kbytes of data.

Recent multibeam wus are 704KB which contains about 693KB of telescope data encoded in "x-setiathome" format. You can actually read the wu file with an XML reader. Assuming these data is in a base-64 encoding, the 8-bit representation of telescope data is about 520KB. From server status page, an average Arecibo tape is 50.20GB, which can be divided into about 101 thousand unique workunits or 202 thousand workunits that require double-checking. A good GPU can finish a workunit in a minute or 1440 workunits per day. We will need about 140 such GPUs to complete 1 Arecibo tape in a day, and 560 such GPUs to complete 4 Arecibo tapes(aa, ab, ac, ad) per day. Refinements are welcome.

Another than a couple of CPU apps the rest are all 32-bit based.

Cheers.

We are currently reporting 130.000 results/hour. (Which is still not enough to do real time crunching)

Considering a top of the line GPU doing 1result/min:

130000/60= 2167

So, 2200 RTX 2080ti GPU's would still not be enough

---

So,
to get the work done the WUs should not be dispersed to the million CPUs but rather to the 10 000 top GPUs.
Set the max 100 per CPU to 10 per CPU and the max per GPU to reflect thrice the daily output.

JMwT (Just My wine-smelling Thoughts)

Petri33

---

To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones

I think there have been a lot of similar comments about the scheduler limit lately and one of the common threads is that work should be attributed to RAC for a host. A simple solution to account for a Android phone that crunches a WU in a couple days and has a very low RAC compared to the multicpu/multigpu hosts with RAC of several 100K.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

I think there have been a lot of similar comments about the scheduler limit lately and one of the common threads is that work should be attributed to RAC for a host. A simple solution to account for a Android phone that crunches a WU in a couple days and has a very low RAC compared to the multicpu/multigpu hosts with RAC of several 100K.

Hi Keith,

But a host with 100K+ RAC is already getting a lion's share more WUs than an Android device. My tablet (Android) has one WU and it's the one it's working on. My Pis crunch more than the tablet yet they only have maybe a dozen WUs in their cache each. My PC has 99 I think.

Perhaps they could raise the 100 WU limit to say 200. I don't believe doling per device RAC would be fair to low RAC devices since they are getting only a few as it is. What happens when SETI has a shortage? Low RAC devices get none, or maybe one here and there. See what I mean? I doubt my Pis would have a dozen in cache during a shortage.

Although, I could just be blowin' smoke since I have no idea how the mechanics of the process works. ;)

Have a great day! :)

Siran

---

CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath

...the RAC chasers cry in their ale, the rest of us just shrug our collective shoulders and think of the money we are saving on our energy bills.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

I think for Arecibo tasks, our capablility is enough to do real time crunching. However there is more GBT data to do. GBT generates many more tapes than Arecibo (Why?) per night. I searched in my job log for a completed GBT night. On Thu Oct 04 2018, I finished task blc22_2bit_guppi_58340_46097_HIP4288_0052.11318.0.20.29.42.vlar_0, which is the first blc22/blc23_2bit_guppi_58340_* task I recieve. Then on Sat Nov 03 2018 I finished task blc23_2bit_guppi_58340_69873_HIP45247_0120.8535.818.19.28.131.vlar_0, which is the last blc22/blc23_2bit_guppi_58340_*_vlar_0 task I recieve. There is one more blc23_2bit_guppi_58340_51498_HIP25486_0067.10241.0.19.28.62.vlar_2 task a month later on Mon Dec 17 2018 (the part vlar_2 means that task was sent to a computer that never returns). So, 1 month of crunching to finish 1 night of data.

How long ago have we been crunching tapes from day 58406? (15 October 2018)

30 days?

Does the project need 30x the current crunching power?

---

Not really - Data collection is far from continuous and the tapes are very often not run in the same sequence as they were collected (remember they are are often pre-processed and screened before we get them)

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

How long ago have we been crunching tapes from day 58406? (15 October 2018)

30 days?

Does the project need 30x the current crunching power?

I see I posted in this thread in January that we were working on this day of data from before then. We have also done a ton of Aricebo data also, so it is hard to get a handle on how long it is taking us, since we haven't been working on 58406 exclusively.

The answer is not a trivial or simple "It takes x hours to process one task (work unit), there are x-tasks worth of data collected per hour" type answer as it really does depend on what the telescope was doing during that 24 hour period.
It cannot observe a single location for 24 hours - the Earth gets in the way for at least 12 hours, meaning the dish has to be repositioned at least once. Then there are other user of the telescope to consider, how many receivers and receiver channels are active. And what "style" of observation is being made (e.g. just staring at a spot vs. jittering around a spot). How long the target is in sight and observable (I don't know if the GBT can "flip" through 90 elevation or not).

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

How long the target is in sight and observable (I don't know if the GBT can "flip" through 90 elevation or not).

The GBT is an alt-azimuth mount so no need to flip the meridian.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)