basically this: it does like 300,000 credits/day

Thank you for the picture.

Now I get it. That "partition" in the middle of the case is the fans "drawing air from the gpus" that you referenced.

It must have hit a "bump" its doing "only" 150,000+- RAC at the moment ;)

And its not listing the Gtx 1080Ti at all?


Tom

RAC is an average daily production over the course of several weeks (4-6 or so).

this system has only been online for 6-7 days. so it will take some time to build up the RAC value.

but you can see the daily production here : https://boincstats.com/en/stats/-1/host/detail/337775086/lastDays

and with the stats page, it doesnt list out every card in the system, it takes whatever is the first card, then just adds a number value for total number of cards in the system. it doesnt account for different types of cards in the same system

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

this system has only been online for 6-7 days. so it will take some time to build up the RAC value.

Generally 6-8 weeks if there are no Seti server issues in that time.

---

Grant
Darwin NT

Wouldn't a NVMe disk (or two) be better than SATA ?
The EVGA X299 Dark has room for 2 of them...I know because I have that mobo.
And btw...would you recommend 1 large disk, or 2 smaller ? Like 1 for the OS and one for apps ?

It really does depend on how much data you have, and how often you access it.

Generally these days if i had a lot of data, i'd go for 2 SSDs. One for the OS & applications etc, the other for data. If a had a real lot of data, 2 SSDs, and 1 or more 4TB+ HDDs.
1TB+ NVMe SSDs are becoming quite affordable, and 2TB+ SATA SSDs even more affordable.

On my present systems my main one has a 1TB NVMe drive, the other a 500GB NVMe drive and a 4TB HDD as an archive. If i were still mucking around with dBase databases i'd have a second 1 or 2TB NVMe SSD just for the database data & a 4TB+ HDD for local backup/ bits and pieces storage on my main system.

---

Grant
Darwin NT

Tnx, Grant.
Just got two Samsung 970 EVO NVMe disks for my new system yesterday, as they were on sale.
In addition I have two large SATA SSDs and an oldfashioned 2TB spinner for file storage.

---

Humans may rule the world...but bacteria run it...

basically this: it does like 300,000 credits/day

So all the video cards in the second bay are sitting on serially linked riser cards? Got a URL for where to buy those?

Thanks,
Tom

---

A proud member of the OFA (Old Farts Association).

basically this: it does like 300,000 credits/day

So all the video cards in the second bay are sitting on serially linked riser cards? Got a URL for where to buy those?

Thanks,
Tom

Looks something like this. 1st google result.
https://www.amazon.com/Fixable-Adapter-Flexible-Extension-Connector/dp/B00IMYODGS
Standard mining USB 1x riser adapter.

i have these: https://www.amazon.com/EXPLOMOS-Graphics-Extension-Ethereum-Capacitors/dp/B074Z754LT/ref=sr_1_fkmr1_2?s=electronics&ie=UTF8&qid=1534903222&sr=1-2-fkmr1&keywords=v8+risers

the 6-pin plug is the best thing to use for powering the riser itself. remember, the card slot can pull up to 75W by itself.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

basically this: it does like 300,000 credits/day

So all the video cards in the second bay are sitting on serially linked riser cards? Got a URL for where to buy those?

Thanks,
Tom

Looks something like this. 1st google result.
https://www.amazon.com/Fixable-Adapter-Flexible-Extension-Connector/dp/B00IMYODGS
Standard mining USB 1x riser adapter.

I see that, but is it possible to serially link that one? If not, you would be limited to the number MB slots that could drive video cards. I was told there is a version that you plug into a single video card slot and then plug each card into the previous card?

Tom

---

A proud member of the OFA (Old Farts Association).

I’m not aware of any risers that are connected in serial like that.

Most risers plug directly into a PCIe slot.

But there are ways to get more cards on a single board.

M.2 uses a PCIeinterface and you can use M.2 to PCIe adapters there.

And you can use some 4 in 1 adapters that allow you to plug 4x GPUs into a single PCIe x1 interface. I’m actually using one of these for my thrown together SETI rig based on my gaming desktop having a mATXboard.

https://i.imgur.com/oSadrfA.jpg

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Thanks for listing these specs. I'm attempting to replicate your machine fairly closely with a company that is building the server for me. One question they had:

"I used the single root complex version so all of the GPUs are connected to one CPU. I can have half of them on one CPU and the other on the second CPU if you’d like. The way people want it depends on their applications and how much GPU to GPU traffic there would be."

Can you advise on what would be optimal here?

Thanks for listing these specs. I'm attempting to replicate your machine fairly closely with a company that is building the server for me. One question they had:

"I used the single root complex version so all of the GPUs are connected to one CPU. I can have half of them on one CPU and the other on the second CPU if you’d like. The way people want it depends on their applications and how much GPU to GPU traffic there would be."

Can you advise on what would be optimal here?

Theres no GPU to GPU communication here. It'd be best to split GPUs between CPUs if they can. Otherwise each could would get fewer lanes but PCI-E bandwidth doesn't seem as big of a deal at SETI. Other projects need more.

Are they using some PCI-E backplane vs plugging them into PCI-E slots with riser cards?
https://www.servethehome.com/single-root-or-dual-root-for-deep-learning-gpu-to-gpu-systems/

I'm not sure what he means by he's hooking all the GPUs to a single CPU.

can you tell me some specifics on what CPUs and what motherboard is being used?
some multi socket boards have the PCIe slots split up between different CPUs, so you cant use certain physical sockets if you don't have both CPUs installed. but that would depend on the motherboard configuration.

but SETI doesn't really care. I have 4 GPUs crammed into a single PCIe x1 slot (a 4 in one card using a PLX chip) and it still doesn't affect the speed.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Thanks for listing these specs. I'm attempting to replicate your machine fairly closely with a company that is building the server for me. One question they had:

"I used the single root complex version so all of the GPUs are connected to one CPU. I can have half of them on one CPU and the other on the second CPU if you’d like. The way people want it depends on their applications and how much GPU to GPU traffic there would be."

Can you advise on what would be optimal here?

It sounds like you maybe setting up a dual cpu MB (two separate sockets)? Yes, you would want to distribute them equally but its not a "kill" issue on Seti. There might be other Boinc apps that it could be.

I have now read the "single root" vs. the other one. Basically Boinc doesn't do that kind of communications between cpus so its not an issue. As far as I know.

My reading of my dual cpu MB seems to indicate that some gpu slots are slowed down depending on what is plugged in where. But I think I understand that Seti gpu processing works nicely even on a PCi-e 1x speed slowed slot no matter what is its actual length.

Plus if you don't have enough to go around, the card mentioned earlier in the thread that allows one slot to drive four gpu's should take care of the issue.

Based on what I am reading, right now, you should not plan on more than 16 gpus split equally between Nvidia and Amd/Ati. That is true based on 1 cpu or 2.

HTH,
Tom

---

A proud member of the OFA (Old Farts Association).

So what kind of specs are you planning for this Server?

Thank you.

Tom

---

A proud member of the OFA (Old Farts Association).

[url]but you can see the daily production here : https://boincstats.com/en/stats/-1/host/detail/337775086/lastDays

How did you find that url? I am curious to see what mine are doing.

Thank you,
Tom

---

A proud member of the OFA (Old Farts Association).

[url]but you can see the daily production here : https://boincstats.com/en/stats/-1/host/detail/337775086/lastDays

How did you find that url? I am curious to see what mine are doing.

Thank you,
Tom

click on your list of computers from your main profile page. under each host will be a boinc stats link like that.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

[url]but you can see the daily production here : https://boincstats.com/en/stats/-1/host/detail/337775086/lastDays

How did you find that url? I am curious to see what mine are doing.

Thank you,
Tom

click on your list of computers from your main profile page. under each host will be a boinc stats link like that.

I am logged into Boincstats and it is not showing my hosts at all. Let me go see if I can find some kind of tutorial.

Tom

---

A proud member of the OFA (Old Farts Association).

if they are new, it will take a couple days to pop up in the stats.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

I ended up using a server company to build a system based on the feedback i received in this thread. (8k total spend instead of original 20k budget :)

One question from the server company i'm not sure how to answer: "Is there a particular way you’d like the GPUs to be installed with respects to the CPUs? Or balanced out?"

Not even sure what this means, or if there is an optimal way to do this for SETI? Thanks!

I ended up using a server company to build a system based on the feedback i received in this thread. (8k total spend instead of original 20k budget :)

One question from the server company i'm not sure how to answer: "Is there a particular way you’d like the GPUs to be installed with respects to the CPUs? Or balanced out?"

Not even sure what this means, or if there is an optimal way to do this for SETI? Thanks!

I think I might understand the question but I am pretty sure the answer is:

1) Make the Gpu's as independent as possible from each other.
2) Provide as much "space" and cooling as is possible.
3) I think the final answer is "balanced out" because you want each cpu to be able to apply as much or as little "horsepower" as you want to set with the assorted parameter files.
4) I am not certain what this means in terms of Bios settings. Anyone?
5) I am not certain what this means in terms of the PCIex speed settings either.

There is discussion but not necessarily complete agreement over how much Video card bus speed programs like the SOG task need because they are hybrid cpu/gpu processing tasks. This may or maynot be true of the Linux/CUDA90+ apps.

I don't know if it will help but you can explain to the server company that this is a numerically oriented processing system (aka: Number Cruncher) where each task being processed is not dependent on any other processed task with the exception of cpu->gpu dependencies. One or more cpu threads are needed for each gpu.

At this level of Seti processing I am pretty sure this is an accurate representation of what we do, and how we do it. Lets see if anyone else comments though.


Tom

---

A proud member of the OFA (Old Farts Association).