inconclusive / invalid / error wu stats

Message boards : Number crunching : inconclusive / invalid / error wu stats
Message board moderation

To post messages, you must log in.

AuthorMessage
W3PerlProject Donor
Volunteer tester

Send message
Joined: 29 Apr 99
Posts: 159
Credit: 1,088,525,114
RAC: 1,976,644
France
Message 1900520 - Posted: 11 Nov 2017, 13:45:40 UTC

I would like to monitor inconclusive / invalid / error for each computer to generate a graph over time.
It will allow to check if something is going wrong and see any changes over app version.

The only way I've found is to extract the data from the setiathome web page. An XML file
would be better and safer. Do you know if such a file exist for each user ?
ID: 1900520 · Report as offensive     Reply Quote
Profile Jeff BuckSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1435
Credit: 147,366,816
RAC: 187,536
United States
Message 1900575 - Posted: 11 Nov 2017, 20:23:03 UTC - in response to Message 1900520.  

That info might be in the hosts file from the daily stats dump, but there aren't separate files for each user, so even if the data is there, you'd have to download and extract it from a massive file each day. That file is too big for me to even look at to see if the data you want is there, so you're own your own to check it out. I think your current method might still be the best.
ID: 1900575 · Report as offensive     Reply Quote
W3PerlProject Donor
Volunteer tester

Send message
Joined: 29 Apr 99
Posts: 159
Credit: 1,088,525,114
RAC: 1,976,644
France
Message 1900598 - Posted: 11 Nov 2017, 21:31:09 UTC - in response to Message 1900575.  

That info might be in the hosts file from the daily stats dump, but there aren't separate files for each user, so even if the data is there, you'd have to download and extract it from a massive file each day. That file is too big for me to even look at to see if the data you want is there, so you're own your own to check it out. I think your current method might still be the best.


Thanks. I didn't know there is an available list of hosts/users from the database. You could see who was the first to register seti@home in 1999 ! ;)
Unfortunately, the file contains only data which can be seen this way : https://setiathome.berkeley.edu/show_host_detail.php?hostid=<id>
using zgrep -A 30 '<id>YOUR_HOST_ID' host.gz
ID: 1900598 · Report as offensive     Reply Quote
Profile Jeff BuckSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1435
Credit: 147,366,816
RAC: 187,536
United States
Message 1900605 - Posted: 11 Nov 2017, 22:31:21 UTC - in response to Message 1900598.  

Well, at least you were able to eliminate that file as a possible source for your information. i can't think of any other existing files, so grabbing the numbers from the first page of each host's task list seems like your only choice.
ID: 1900605 · Report as offensive     Reply Quote
Profile Keith MyersSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 3146
Credit: 203,863,494
RAC: 288,628
United States
Message 1900607 - Posted: 11 Nov 2017, 22:40:46 UTC - in response to Message 1900605.  

Doable if he had half a dozen hosts. But I think Laurent has over a hundred.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1900607 · Report as offensive     Reply Quote
Profile Jeff BuckSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1435
Credit: 147,366,816
RAC: 187,536
United States
Message 1900609 - Posted: 11 Nov 2017, 22:46:16 UTC - in response to Message 1900607.  

Doable if he had half a dozen hosts. But I think Laurent has over a hundred.
Nothing that one of his Perl scripts can't handle. ;^)
ID: 1900609 · Report as offensive     Reply Quote
W3PerlProject Donor
Volunteer tester

Send message
Joined: 29 Apr 99
Posts: 159
Credit: 1,088,525,114
RAC: 1,976,644
France
Message 1900780 - Posted: 12 Nov 2017, 21:30:09 UTC - in response to Message 1900609.  

Doable if he had half a dozen hosts. But I think Laurent has over a hundred.
Nothing that one of his Perl scripts can't handle. ;^)


Thanks ;)
Btw, I have updated the cpu2gpu.pl script (fix a problem when no more GPU wu are available).
I'm writing some perl scripts to monitor RAC over mutiple hosts on linux (it retrieve client_state.xml from nfs and produce some graphs
sorted by CPU or GPU). I'm adding inconclusive/invalid/error stats....
If anyone running several computers on linux is interested, let me know...no problem to share !
ID: 1900780 · Report as offensive     Reply Quote
Profile Brent NormanSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1963
Credit: 131,237,326
RAC: 344,850
Canada
Message 1900830 - Posted: 13 Nov 2017, 5:07:40 UTC - in response to Message 1900780.  

I'm interested :)
Are you going to update your download page, or are these one-offs?
ID: 1900830 · Report as offensive     Reply Quote
W3PerlProject Donor
Volunteer tester

Send message
Joined: 29 Apr 99
Posts: 159
Credit: 1,088,525,114
RAC: 1,976,644
France
Message 1900933 - Posted: 13 Nov 2017, 19:12:25 UTC - in response to Message 1900830.  

I'm interested :)
Are you going to update your download page, or are these one-offs?


I will pm you once the script will be more generic (the list of host is in my own database).
This is a alpha script right now.
If you send me your email, I can send you a demo (which should stay private) and you'll be able to
send me feedbacks.
ID: 1900933 · Report as offensive     Reply Quote
Profile Brent NormanSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1963
Credit: 131,237,326
RAC: 344,850
Canada
Message 1901013 - Posted: 14 Nov 2017, 3:41:37 UTC - in response to Message 1900933.  
Last modified: 14 Nov 2017, 3:44:01 UTC

Very cool page you have there :D

I can see it being useful to you with a large amount of computers, but for the rest of us ... I don't know. Most of us can count our computers on one hand, maybe need two - you have to invite the whole family and a few relatives to count them all :)

Feedback:
- Percentage for (In progress, Validation pending, Validation inconclusive, Valid, Invalid, Error) is nice, but would be nice to know per computer as well, not just the total tasks. Maybe it's in there somewhere, but I don't see it.
- Histo and RAC Delta I am sure are useful, if I knew how to read them.
- Your 'All Computers' history graph looks cool with 110 computers on it. It is the same as the BoincTasks graph shows
- Total credits/RAC per computer/user are also in BoincTasks
- I don't think your coding works with multi card machines - which most of us have. You have a dual 1070 in your computer list (only one I could find on SETI list) and it doesn't show up on this page - likely intentionally since it has dual cards. I could see it being a programming nightmare to separate them like you have all your single card computers on the page. Which brings us back to only having up to 10 computers - we wouldn't need them all categorized by CPU/GPU like you have done for yourself.

So for us I could see a useful page having:
- Percentage Total and per computer ID
- RAC per computer graph
- Histo and Delta
EDIT: I doubt we need reading computer info into a database, since 10 computers entries in the script wouldn't be hard at all.
ID: 1901013 · Report as offensive     Reply Quote
Profile Brent NormanSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1963
Credit: 131,237,326
RAC: 344,850
Canada
Message 1901099 - Posted: 15 Nov 2017, 2:31:36 UTC

Perl sent me a link to his Alpha page for monitoring his computers and asked me not to give it out, but he didn't say I couln't show a screen shot of it :D
110 Computers Graphed by RAC
There is a lot more to show sorted by card or CPU type.
ID: 1901099 · Report as offensive     Reply Quote
W3PerlProject Donor
Volunteer tester

Send message
Joined: 29 Apr 99
Posts: 159
Credit: 1,088,525,114
RAC: 1,976,644
France
Message 1901589 - Posted: 17 Nov 2017, 22:56:23 UTC - in response to Message 1901013.  



Feedback:
- Percentage for (In progress, Validation pending, Validation inconclusive, Valid, Invalid, Error) is nice, but would be nice to know per computer as well, not just the total tasks. Maybe it's in there somewhere, but I don't see it.


Not yet but will add also per computer.


- Histo and RAC Delta I am sure are useful, if I knew how to read them.

To check how many computer have their RAC increase or decrease over the last month.


- Your 'All Computers' history graph looks cool with 110 computers on it. It is the same as the BoincTasks graph shows
- Total credits/RAC per computer/user are also in BoincTasks


Sadly BoincTasks run on Windows...I've tried with wine but without success....



- I don't think your coding works with multi card machines - which most of us have. You have a dual 1070 in your computer list (only one I could find on SETI list) and it doesn't show up on this page - likely intentionally since it has dual cards. I could see it being a programming nightmare to separate them like you have all your single card computers on the page. Which brings us back to only having up to 10 computers - we wouldn't need them all categorized by CPU/GPU like you have done for yourself.


This machine is located on another network...need to find a way to retrieve the stats file remotely.

Thanks for your comments ! :)
ID: 1901589 · Report as offensive     Reply Quote
Profile Brent NormanSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1963
Credit: 131,237,326
RAC: 344,850
Canada
Message 1901595 - Posted: 17 Nov 2017, 23:47:56 UTC - in response to Message 1901589.  

I was thinking yesterday about that Percentage thing ... Since you are already collecting data to a database and graphing it, why not do it for percentages as well. Application changes would likely be easier to spot that way.

Sadly BoincTasks run on Windows...I've tried with wine but without success....

I'm not sure what you use as 'your' computer, but I see many Ubuntu 14 computers on your list (I think they are anyways)

On Ubuntu 14 I installed:
- Wine Windows Program Loader
- PlayOnLinux

BoincTasks complained about a language library on each restart (requiring a reinstall each launch) but copying the file it was looking for to where it wanted it, cured that. Reordering columns doesn't work quite right, but haven't looked at it, it's not a big deal for me.

Other than that it has been working fine, but a little slower response than Windows.
But that may be due to the high GPU use on this 3x1080 machine.

I haven't tried any other Windoze Apps.
ID: 1901595 · Report as offensive     Reply Quote

Message boards : Number crunching : inconclusive / invalid / error wu stats


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.