4 x 2080 ti gpu cards

Message boards : Number crunching : 4 x 2080 ti gpu cards
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Miklos M.

Send message
Joined: 5 May 99
Posts: 955
Credit: 136,115,648
RAC: 73
Hungary
Message 1995477 - Posted: 27 May 2019, 16:45:58 UTC

Is this possible? Has anyone tried it and how much cooling and power I need?

Thank you
ID: 1995477 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995478 - Posted: 27 May 2019, 16:53:19 UTC - in response to Message 1995477.  

Closest host I could find is Tod's with (3) X 2080 Ti plus a Titan V. The power requirements would require a minimum 1200W or greater PSU. You can always limit the power on the cards to 200W and it doesn't reduce the crunching speed that much.
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8634807
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995478 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995484 - Posted: 27 May 2019, 18:22:35 UTC

Just got chastised by a team mate that he is running four RTX 2080 Ti. He recently joined the one million RAC SETI project club.
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8687856
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995484 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1995486 - Posted: 27 May 2019, 18:52:14 UTC

As far as cooling. It’s definitely something that has to be considered carefully. I believe Mr Kevvy has them watercooled. So being in close proximity to the other cards isn’t a huge issue.

But if you are air cooling I would try to separate the cards using risers.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1995486 · Report as offensive
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 3832
Credit: 1,114,826,392
RAC: 3,319
Canada
Message 1995487 - Posted: 27 May 2019, 18:59:35 UTC
Last modified: 27 May 2019, 19:21:59 UTC

LoL it's OK, Keith... hard to tell with the inflated GPU count. :^)

The machine Keith linked to is indeed 4 x RTX 2080tis, specifically AORUS GeForce RTXâ„¢ 2080 Ti XTREME WATERFORCE WB 11G. They're on a custom loop so that they can blow their heat out a window rather than into our apartment that swelters already in the summer.

The rad is an MO-RA3 360 LT:



I mounted it on a board with eight heavy carriage bolts and fender washers. Rather than nine wimpy computer fans, I hung one of these on it upside-down in "push" configuration with a threaded hook through the board and more fender washers:



This fan (designed and built in the same metropolitan area I live) is well-known for being designed like a jet engine and producing a silent, laminar flow of air. I also chose it because replacements are easy to find and reinstall, being just hung there. And I have so far have only had to run on the lowest speed "1" to keep the system cool... if things heat up in the summer there's still "2" and "3".

The primary PSU is an eVGA SuperNOVA 1300 G2 120-G2-1300-XR 80+ GOLD 1300W‎ which does everything except two of the cards. The other two cards are powered by a 1000W helper PSU zip-tied into the case in the peripheral bay. This also allows the load to be split over two circuits... also essential in an apartment! They are joined via a 24Pin 20+4pin Dual PSU ATX Power Supply Adaptor Cable aka PSU ganging cable ($5 on Fleabay.) I did this as these cards can draw 260W apiece at full-bore, so the whole system would be 1200W+, too close to the margin for one 1300W PSU (I like to keep it below 2/3 of the rated capacity long-term, and don't tend to burn out PSUs much by sticking to this rule-of-thumb.)

Pump is just a standard EKWB 140mm pump/res combo. Most would say the res. is too small for the amount of tubing I have but by being careful with elevations of the PC/rad. when filling I had no issues. Highest coolant temp. I have seen is 37C and the GPUs are below 50C at that time.

One caveat: I have not been able to successfully overclock those cards. Cinnamon crashes as soon as there is an xorg.conf and even overclocking very lightly, and not on the display card, it produces invalids. Could not stop them so gave up on overclocking. I also found out that accidentally unplugging the secondary PSU will also make invalids, rather rapidly!

Hope this helps. :^) Sorry no pics of the actual setup yet... keep forgetting to upload them. I may edit them in when I get home.
ID: 1995487 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1995496 - Posted: 27 May 2019, 20:13:27 UTC - in response to Message 1995484.  

Just got chastised by a team mate that he is running four RTX 2080 Ti. He recently joined the one million RAC SETI project club.
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8687856



If he hadn't I would have after reading the article about his flagship, lol....
ID: 1995496 · Report as offensive
Miklos M.

Send message
Joined: 5 May 99
Posts: 955
Credit: 136,115,648
RAC: 73
Hungary
Message 1995497 - Posted: 27 May 2019, 20:14:19 UTC - in response to Message 1995478.  

Thank you very much for the detailed support information.
ID: 1995497 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995498 - Posted: 27 May 2019, 20:33:40 UTC - in response to Message 1995487.  

There must be many differences for the Cinnamon DE compared to the GNOME DE and the reason that you can't get xorg.conf to work in Cinnamon.

Anybody else using Cinnamon DE and getting overclocking to work?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995498 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 37564
Credit: 261,360,520
RAC: 489
Australia
Message 1995535 - Posted: 27 May 2019, 23:00:06 UTC

One caveat: I have not been able to successfully overclock those cards. Cinnamon crashes as soon as there is an xorg.conf and even overclocking very lightly, and not on the display card, it produces invalids. Could not stop them so gave up on overclocking.
I'm having the same problem and all I want is access to fan control. :-(

When I get some free time I'll dive back into the Mint forums and see what I can find out.

Cheers.
ID: 1995535 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995542 - Posted: 27 May 2019, 23:23:58 UTC - in response to Message 1995535.  

Wiggo, can you try these commands from the Terminal and see what happens.

/usr/bin/nvidia-smi -pm 1

/usr/bin/nvidia-smi -acp UNRESTRICTED


If you get a good reply from the unrestricted and persistence mode, then try this:

/usr/bin/nvidia-settings -a "[gpu:0]/GPUPowerMizerMode=1"
/usr/bin/nvidia-settings -a "[gpu:0]/GPUFanControlState=1"
/usr/bin/nvidia-settings -a "[fan:0]/GPUTargetFanSpeed=100"

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995542 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 37564
Credit: 261,360,520
RAC: 489
Australia
Message 1995548 - Posted: 27 May 2019, 23:48:33 UTC - in response to Message 1995542.  

Wiggo, can you try these commands from the Terminal and see what happens.

/usr/bin/nvidia-smi -pm 1

/usr/bin/nvidia-smi -acp UNRESTRICTED


If you get a good reply from the unrestricted and persistence mode, then try this:

/usr/bin/nvidia-settings -a "[gpu:0]/GPUPowerMizerMode=1"
/usr/bin/nvidia-settings -a "[gpu:0]/GPUFanControlState=1"
/usr/bin/nvidia-settings -a "[fan:0]/GPUTargetFanSpeed=100"
I'll give that a go Keith when I get a chance (hopefully later today as I have a bit of work going on here ATM, but maybe tomorrow morning) and let you know what happens.

Cheers.
ID: 1995548 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1995557 - Posted: 28 May 2019, 0:33:09 UTC - in response to Message 1995487.  

That rad setup is identical to mine. I have it mounted in a window also with a big fan on it haha. Works great.

But I’m using dual D5 pumps for redundancy and some extra oomph.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1995557 · Report as offensive
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 3832
Credit: 1,114,826,392
RAC: 3,319
Canada
Message 1995839 - Posted: 29 May 2019, 20:59:34 UTC - in response to Message 1995557.  
Last modified: 29 May 2019, 21:08:42 UTC

That rad setup is identical to mine. I have it mounted in a window also with a big fan on it haha. Works great.


Ahh... it was your photo of that setup that inspired me to go with the big rad/fan; wasn't sure as never saw it again. I had not seen such a large rad for PC use before, so was not aware they were made that large. Not only large, but when I first pulled that gorgeous hunk of metal out of the box I was astounded how thick, solid and well-made it was, so not a bad price plus considering it was imported from Germany, where there is certainly quality but not cheap.

Here's the rad/fan setup:



And the computer itself:



Not pretty, but it works. :^p Note the secondary PSU on the right Zip-tied in the peripheral bay. The clear thing on the tubing upper-left is a flow indicator. There's also a temp. indicator above the primary PSU hiding behind cabling. And I decided against liquid-cooling the CPU so left the existing Noctua cooler in place, as I didn't know if I would get sufficient cooling for the cards from this setup (plus why waste a good Noctua and spend more for an unneeded CPU block.) I did have to cut away some of the rad fins to run the tubing.

Looks like the coolant needs topping up there... there is a very slow loss which I've been unable to find, and not from lack of trying, a mm or two daily. (I had to improvise a filler with a silicone reusable drinking straw and tiny funnel, as there's very little clearance under the hard drive.) The second-from-the-bottom card LEDs don't work; this is probably why it was an open-box return that was sold to me as new that took weeks to unregister via Gigabyte, but I would actually prefer they were all off. And I need to still get the right large case fan working before summer... had to order a Molex to fan connector adapter as the cord is too short.

Hope I can keep it running. :^)
ID: 1995839 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995845 - Posted: 29 May 2019, 21:30:35 UTC - in response to Message 1995839.  

Thanks for the pics Mr. Kevvy. I really like the lab pipette squeeze bottles with the angled neck for filling the reservoir. Your coolant loss probably isn't loss but rather entrained air finally getting displaced so the level in the reservoir is appearing to go down.

I would think that radiator has plenty more of wattage dissipation so you should be able to add the cpu into the loop if you ever desire.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995845 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1995880 - Posted: 30 May 2019, 2:09:18 UTC

I haven't had any problems with Mint 17.3 or 19 with CoolBits.
ID: 1995880 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995888 - Posted: 30 May 2019, 3:02:59 UTC - in response to Message 1995880.  

Brent, could offer some suggestions for Wiggo. I am unfamiliar with Mint.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995888 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1995889 - Posted: 30 May 2019, 3:10:20 UTC - in response to Message 1995845.  

he'll definitely have a small amount of fluid loss over time. the water evaporates and slowly seeps out. and with higher loads and liquid temps, it just accelerates it. i see a gradual fluid loss on my setup also, have to top it up ever couple months.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1995889 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995894 - Posted: 30 May 2019, 3:46:25 UTC

I guess I should look at my custom loop rig and see where the levels are. I haven't looked at it since putting it together last year.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995894 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1995895 - Posted: 30 May 2019, 3:48:38 UTC - in response to Message 1995894.  

Blow out your rads while you are doing that.....
ID: 1995895 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995898 - Posted: 30 May 2019, 4:05:40 UTC - in response to Message 1995895.  

That is a lot more work than just refilling. I need to go out to the garage and get the appliance dolly to move it out to the garage to the compressor. Chassis is too heavy to carry that far. I did the X9 chassis a couple of days ago. I agree the X5 chassis probably needs it just as bad.

The gpu loop had no change in water levels in the reservoir. The cpu loop was down about 3 mm from where it was. I still think the change was from entrained air getting purged finally. I don't think the PrimoChill Primoflex LRT tubing is permeable at all.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995898 · Report as offensive
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : 4 x 2080 ti gpu cards


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.