Days of future past, or what would you like to see in SETI 2.0

Message boards : Number crunching : Days of future past, or what would you like to see in SETI 2.0
Message board moderation

To post messages, you must log in.

AuthorMessage
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 2044759 - Posted: 14 Apr 2020, 18:49:10 UTC

Ok, I know, it might be too soon, as the corpse isn't even cold yet, but ever since I came to the realization (and finally, acceptance) that it was actually... over (at least for now), I started thinking about what's next for the project? I understand that they have a ton of data to process, and as computing technology seems to accelerate yearly, hopefully it won't take anywhere Near 20 years to process the 20+ that they have collected.

No idea what that time frame might be, but lets toss out say, a year or so? In that downtime, I was thinking that this might be a good time, kind of like this Corona thing going around has done, to re-evaulate things, (like work from home for more ppl on a regular basis after it has passed?) and maybe update/upgrade how SETI works.

I am not up on exactly how many of the issues we have with it as it is now (weekly down times for DB work, etc) are SETI related as opposed to BOINC related, but considering that SETI is the granddaddy of them all, I'm absolutely certain that there are more than a couple legacy issues, just due to it's longevity.

Just as you really can't repair a jet plane engine while it's in flight, doing a major rework to SETI (again that is, if the main issues we've had/have are SETI not BOINC) while the project is up and active, is Really hard to do. I know they have been keeping it up and running, and even doing some upgrades over the years, but until now with it being basically down, or at least soon to be so, maybe it's time to start the discussion as to how it might be "updated/modernized" into the 2020's?

This of course is allowing for the belief that there will still be a SETI in the future. I personally hope that there is, and I know that people much smarter than I would have background knowledge as to the processes and such of how things currently work, and what changes might be possible, or even desirable for the future 'upgraded' version.

Seeing that at this point it is just a thought experiment, toss out whatever ideas you might have as to the design or whatever that you think would be great to have for SETI v2.0!

ID: 2044759 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 12628
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2044768 - Posted: 14 Apr 2020, 19:26:55 UTC

No clue as to how to design the next Seti 2.0 search, but I think that the hardware needs to be upgraded first. And the next mandatory upgrade would be a modern database that scales easily on the new hardware and I would hope that it would be able to handle the amount of transactions we were running up till the beginning of December before the project flubs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2044768 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2044774 - Posted: 14 Apr 2020, 19:48:06 UTC
Last modified: 14 Apr 2020, 19:59:05 UTC

One obvious upgrade must be the size of the WU to be compatible with the processing power of the next generations of GPU/CPU's.
Maybe even a variable size of WU depending of the device who crunch the task.

A big welcomed Upgrade could be DELETE forever CreditScrew! And use a simple but reliable credit score. Something like give the WU a credit independent from what device it was crunched.

Other suggestion is to end the cpu/gpu WU limits. Set the number of WU a host could DL directly proportional to the host crunching capacity. Something like allow a host to DL WU enough for 1 or 2 days max and BTW make the datelines compatible with that. If the host returning only error, no valid work or not work at all, penalize it by stop sending new work until that was fixed.
ID: 2044774 · Report as offensive     Reply Quote
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 961
Credit: 537,293
RAC: 9
United Kingdom
Message 2044778 - Posted: 14 Apr 2020, 20:03:16 UTC

Tasks that run for at least 30 minutes on the fastest crunchers, but no more than 7-14 days on the slowest

Deadlines that expect an average of 4-8 hours per day crunching on slower hosts

BTW, I think this will be more like V 10.0 than 2.0
ID: 2044778 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13181
Credit: 208,696,464
RAC: 304
Australia
Message 2044812 - Posted: 14 Apr 2020, 23:08:33 UTC - in response to Message 2044768.  
Last modified: 14 Apr 2020, 23:14:37 UTC

No clue as to how to design the next Seti 2.0 search, but I think that the hardware needs to be upgraded first. And the next mandatory upgrade would be a modern database that scales easily on the new hardware and I would hope that it would be able to handle the amount of transactions we were running up till the beginning of December before the project flubs.
I think it's fair to say most of the issues we have seen are due to hardware limitations (RAM, DIsk I/O, CPU speeds/Cores/threads). Remove the hardware limitations, then we'll see what the software limits are.

One improvement would be with Assimilation. It looks like it runs through the Tables, starting with the oldest to the most recent looking for WUs that can be Assimilated. No problem with plenty disk I/O and the tables & indexes held in RAM. But as we found a problem when things aren't so good. Older Tasks awaiting Assimilation are there due to outstanding Tasks. Start with the most recent ones, they're more likely to have no outstanding Tasks holding up Assimilation. This will help avoid Assimilation backlogs in the future (as would better hardware).


Personally, a fixed deadline is best. However 1 month is plenty- most of the Tasks here were returned within 48hrs. 1 month still allows the slowest of the slow barely capable & barely on system to contribute. It also allows a grace period if someone has system or other issues- eg recent Tornadoes in the US, fires then floods here in Australia. It gives people a chance to get their systems up and running as power & communications are restored & they don't lose work or get penalised for missing an extremely short deadline.
Grant
Darwin NT
ID: 2044812 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13181
Credit: 208,696,464
RAC: 304
Australia
Message 2044813 - Posted: 14 Apr 2020, 23:17:34 UTC - in response to Message 2044774.  

A big welcomed Upgrade could be DELETE forever CreditScrew! And use a simple but reliable credit score. Something like give the WU a credit independent from what device it was crunched.
I've always advocated that they award Credit according to the actual definition of a Cobblestone.
The actual FLOPS counting came closest to it. It wasn't perfect, but it was a damn sight better than anything before or since.
Grant
Darwin NT
ID: 2044813 · Report as offensive     Reply Quote
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 27986
Credit: 53,134,872
RAC: 32
United States
Message 2044830 - Posted: 15 Apr 2020, 0:42:04 UTC

Uh, how about we actually think about things like the antenna, areas of the sky and the frequencies to be searched.

Seti@home has been doing a very narrow search of frequencies primarily from one antenna. It has only been able to search about 40% of the sky on only a few Mhz in bandwidth.

Serendip, searches a very large frequency spread and there are receivers on several antenna, however it is nearly deaf compared to seti@home's deep search.

I think 2.0 should find some way to search 100% of the sky over a very wide frequency spread, including optical, and do so with very sensitive ears. First, I doubt if would run on BOINC.
I'd like to see something where it could do rapid possible signal identification so that additional deep targeted searches could be made in the same patch of sky.
GPU's have made a big difference, but I'm wondering if some sort of purpose built FFT/GPU could be made even faster. Get our crunch force to purchase enough of these that the cost to make the chips is reasonable. If lucky say pin replacement for an existing GPU so we don't have to make cards.

Just dreaming.
ID: 2044830 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13181
Credit: 208,696,464
RAC: 304
Australia
Message 2044833 - Posted: 15 Apr 2020, 0:53:12 UTC - in response to Message 2044830.  

Uh, how about we actually think about things like the antenna, areas of the sky and the frequencies to be searched.

Seti@home has been doing a very narrow search of frequencies primarily from one antenna. It has only been able to search about 40% of the sky on only a few Mhz in bandwidth.
We had data from 2 antennas (GBT & Arecibo), and should have had data from a 3rd- this one in the Southern Hemisphere (Parkes), but unfortunately we didn't get to that point.
But more data sources in the future would be good, but not much point having them if Seti can't keep up with the workload.

The servers were generally good for a 150k/hr return rate, but whenever it got over that then the Scheduler, upload & down load servers would start having issues. If Seti is to come out of hibernation, it needs to be capable of dealing with sustained loads of 350k/hr as an absolute minimum.
Grant
Darwin NT
ID: 2044833 · Report as offensive     Reply Quote
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 27986
Credit: 53,134,872
RAC: 32
United States
Message 2045157 - Posted: 16 Apr 2020, 3:01:05 UTC - in response to Message 2044833.  

Uh, how about we actually think about things like the antenna, areas of the sky and the frequencies to be searched.

Seti@home has been doing a very narrow search of frequencies primarily from one antenna. It has only been able to search about 40% of the sky on only a few Mhz in bandwidth.
We had data from 2 antennas (GBT & Arecibo), and should have had data from a 3rd- this one in the Southern Hemisphere (Parkes), but unfortunately we didn't get to that point.
But more data sources in the future would be good, but not much point having them if Seti can't keep up with the workload.

The servers were generally good for a 150k/hr return rate, but whenever it got over that then the Scheduler, upload & down load servers would start having issues. If Seti is to come out of hibernation, it needs to be capable of dealing with sustained loads of 350k/hr as an absolute minimum.

What I was talking about would be more like 150M/hr, if it was on BOINC architecture. I don't think anything short of a totally new DC platform could keep up.

While we had GBT for a while, GBT was not doing drift searches, it was mostly pointing at selected targets so it covered a minuscule portion of the sky. Parks would have been drift searches and thus covered a huge area as would the SKA. Or you can't know where in advance to look, so you need to look everywhere.

Haven't checked the back of the envelope math, but assume GBT searched 1000 targets and had 1/2 square degree searched per target. There are 64800 square degrees in the sky. That is something like 0.5% of the sky.
ID: 2045157 · Report as offensive     Reply Quote
Filipe

Send message
Joined: 12 Aug 00
Posts: 209
Credit: 21,281,677
RAC: 20
Portugal
Message 2045229 - Posted: 16 Apr 2020, 15:56:36 UTC
Last modified: 16 Apr 2020, 16:01:20 UTC

What I was talking about would be more like 150M/hr, if it was on BOINC architecture. I don't think anything short of a totally new DC platform could keep up.

Something like folding@home is doing could solve this. Use several servers on several locations.
Maybe 1 for Arecibo, 1 for GBT, 1 for Parkes, etc...


Uh, how about we actually think about things like the antenna, areas of the sky and the frequencies to be searched

I agree with you. if current search/algorithms haven'tfind anything over 20 years, the search should be expended to other frequencies.


GPU's have made a big difference, but I'm wondering if some sort of purpose built FFT/GPU could be made even faster.

I don't think anyone would ever want to pay to develop and produce such a dedicated FPGA/ASIC hardware.


BTW, I think this will be more like V 10.0 than 2.0

It has been talked here before, for seti V10.0, a feature would be to change the way work units are produced:
Meaning a WU that has data from the 14 channels in order to identify and remove RFI on the fly instead of doing it on the post-processing Nebula.
Bigger WU with 14 or 28 times the data, will go a long way to reduce load on servers...
ID: 2045229 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2045230 - Posted: 16 Apr 2020, 16:09:01 UTC - in response to Message 2045229.  
Last modified: 16 Apr 2020, 16:13:04 UTC

I agree with you. if current search/algorithms haven'tfind anything over 20 years, the search should be expended to other frequencies.

I only wish to add to this point.
Since the data crunched was never analyzed, there is no way to say if we found or no something "interesting" in this > 20 years.
This is a common mistake, our job is only part of the process, the real search for the gold signal is coming.
Hope in the following months we could have some good news.
ID: 2045230 · Report as offensive     Reply Quote
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 20028
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2045235 - Posted: 16 Apr 2020, 16:25:31 UTC

I agree with you. if current search/algorithms haven't find anything over 20 years, the search should be expended to other frequencies.

As we were only doing the initial sort for potential sources then it is too early to say we haven't found anything yet - for that judgement call we have to wait on the output from Nebula which is still in the alpha stage (develop the algorithms) before deployment on the full data set of however billion potentials we have reduced the initial data set to.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2045235 · Report as offensive     Reply Quote
Gene Project Donor

Send message
Joined: 26 Apr 99
Posts: 149
Credit: 48,393,279
RAC: 118
United States
Message 2047310 - Posted: 27 Apr 2020, 19:57:32 UTC

Seti 2.0 might choose a different kind of "signal" to look for. The present search looks for RF energy in a very narrow bandwidth. I understand the motivation, to improve the s/n ratio by reducing the bandwidth. My thought is to look, instead, at a wide band in the RF spectrum and see if the emitted radiation in that band exceeds the expected natural background level. For example, if the observer is far away and looks at the emission from the earth in the band from 0.1 Ghz to 10 Ghz wouldn't there be an obvious excess as compared to a planet with no technological civilization? How many megawatts are there from all the cell phone devices operating in that band? Not to mention TV and FM broadcast stations. O.K., so the earth's emission is hidden in the sun's (natural noise) emissions and the receiver's spatial resolution maybe can't seperate the two. But, for instance, in the exoplanet business the astrophysicists are detecting the absorption of some small fraction of light, from the total stellar emission, due to the atmosphere of a transiting planet (as compared to the spectrum of the star when the planet is NOT in transit). And, yes, I understand this is measured in a relatively narrow spectral band where water, or carbon dioxide, has a strong effect. Just trying to make an analogy. What if the target "civilization" doesn't use that RF band any more? Maybe everything is fiber optics or millimeter waves. Well, then, Seti 1.0 wouldn't find anything either. In the case where a "planet" is found with an anamalous excess in some spectral region then that's the place (frequency) to look for discrete narrow band signals. How to avoid, or compensate for, the overwhelming noise of the local radio environment? I have no idea! Just brainstorming here...
ID: 2047310 · Report as offensive     Reply Quote
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 708
Credit: 8,032,827
RAC: 62
France
Message 2047378 - Posted: 28 Apr 2020, 8:43:33 UTC
Last modified: 28 Apr 2020, 8:44:15 UTC

https://www.seti.org/seti-institute/project/details/seti-observations remember this, only listening around the Hydrogen Line

in twenty years of listening the space around us in limited lattitudes and frequencies, now somes signals from 20YL distance are juste comming knocking our doors and we shut down the analysing, not the listenning i hope ... but this takes a lot of storage space ... how can they do that ?
ID: 2047378 · Report as offensive     Reply Quote

Message boards : Number crunching : Days of future past, or what would you like to see in SETI 2.0


 
©2021 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.