r/chia•
failburt

How are you managing your large farm?

Farming

After a certain point, it becomes kind of tedious to manage individual drives. I think my patience ends at around 10 drives.

How are you managing 20, 50, or even 100+ hard drives on the software side?

I've love to hear what others are doing to help me decide. Did you make sure to get matching drives to make an orderly ZFS pool? Did you skip the cost of parity drives because plots are among the most replaceable data there is? Or are you actually still managing individual drives and run out of letters in the alphabet for drive names long ago? Did you set up mergerfs to help manage the mess?

Posted Jun 9, 2021, 1:10 AM UTC

231
sparky_903•

giant raid0 array.

3
BlueBird1800•

This has to be a joke

1
aj10017•

Better to just ditch RAID altogether lmao

1
_alexgaribay•

I have 48 disks connected in JBOD enclosures. All of them are together as 1 volume with mergerfs. Two plotters and a farmer mount the volume over CIFS. I use rsync to move completed plots over the network and limit the transfer speed to not oversaturate the connection. I will be adding another 72 disks to the same server.

2
web_dev_tosser•

poorly

2
Silent_Gemini•

26 drives, ~132TB at the moment. Mostly WD but a sprinkle of others with 1 ext. One drive has a letter, the other drives are mounted as a folder under that 1st drive.

No parity, I haven't lost anything in ~4yrs, built it for Burst back in the day.

It has pretty much been set it and forget it for most of its life.

Windows 10

1
[deleted]•

[deleted]

2
Silent_Gemini•

On the file tree, it just looks like a single drive with a bunch of sub-folders.

In my File explorer I just see the one drive with its capacity bar. If I want to check the capacity of the drives mounted as sub-folders I have to do a right-click properties. That's annoying, but I only have to deal with it towards the end of the fill.

When I add a drive, I just physically mount it, plug it in, mount it as the next sub folder name, fill it with plots and never really think about it again.

1
[deleted]•

[deleted]

2
[deleted]•

[removed]

1
AutoModeratorMOD•

This post has been removed from r/Chia because your account has a negative karma score. Please try again when your account has a positive karma score.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

Still trying to acquire 10 drives. I only have 9 with 58tb. No problems so far. All 500 plots under 3 seconds. If your trying to cut down network traffic. Only fill one drive at time. The rest should only be reading. It is silly to worry about one plot not winning. By writing to 100s of drives at the same time will only magnify the issue of read write delays on a network. Hope this helps. Some plots will just have to sit on the sidelines for a while.

1
[deleted]•

[deleted]

1
BlueBird1800•

Once they’re full there’s not so much to manage. You add the drive to the farmer then it just sits there and does it’s thing. Monitor response times in the chia debug log for times >5 sec. I see this time often suggested, but 30 sec is the max before the chia network won’t accept a win.

1
JagaTelesin•

Stablebit Drivepool to aggregate data drives. Snapraid to create soft parity.

1
[deleted]•

[deleted]

2
JagaTelesin•

You'll need at least one drive equal in size (or larger) to the largest drive in your array. i.e. I'm using a bunch of 8TB drives, so I need a parity drive that's 8TB large. If I added a 12TB to the array, I'd need to size my parity drive up to 12TB also.

You can split parity across multiple smaller drives though, which is what I've done with 2x 4TB WD Reds. Allows you to re-use smaller drives if you increase array size later.

Edit: As for not running parity, no - I'll probably always run with parity. Especially the older the drives get. There's more data on the array than just plots. :)

1
[deleted]•

[removed]

1
AutoModeratorMOD•

This post has been removed from r/Chia because your account has a negative karma score. Please try again when your account has a positive karma score.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1
Annh1234•

You get one of these: https://www.supermicro.com/en/products/system/Storage/4U/SSG-640SP-E1CR90

90 drives in 4u with no hassle.

1
sparky_903•

other than the $7000+ price tag sure.

1
Annh1234•

Not sure how much the 90drives one is, but the 60 drive one was about 5k cad a few years back.

And if you were to really use the 60 drives ( 30k+ cad...), it's not that expensive vs the hardware you would hack up together.

Power, data cables, cooling, vibration damping, plus controllers, add up.

Plus it's more power efficient and doesn't take alot of space, and you can add more modules with only drives ( much cheaper ).

After a certain size, you need this kind of stuff, you can't have 1000 drives hanging all over your bedroom. ( If you plan to win anything that is, at 10-20 drives, you can do whatever)

1
justwjx•

I have 4200+ plots connected and mounted in just one NUC, average pass filter time is around 1 second.

1
[deleted]•

[deleted]

1
justwjx•

30+ usb external drives (8tb ~14tb each) directly connected to NUC through 4 Hubs

4 NAS (Qnap / Synology) hold 13 drives share with NFS to NUC (4tb ~ 14tb each)

1 ubuntu desktop hold 8 drives share with NFS to NUC (4th ~ 8tb each)

no complex tools been deployed

1
CyberSKulls•

Once you scale to a certain point you'll want larg disk chassis. I prefer the Supermicro 45 bay JBODs simply because they used to be very affordable and parts were readily available. I run 3 x 45 bays per host (node). Use folder mounts rather than drive letters. Once they are plotted they are essentially set and forget.

Skip any kind of parity or raid. It's a waste of disks for chia.

1
paper_st_soap_llc•

I'm running ~48 drives in 4 12-bay SAS expander boxes, plus another ~24 USB drives, plugged into 4 different PCs. It all adds up to ~300TiB.

I don't do any kind of RAID. Each drive is just a simple drive. That way there are no dependencies. If I want to unplug one and move it to a different host, it's not a problem.

I run all linux, so I don't have any worries about running out of drive letters. I found that after /dev/sdz comes sdaa, sdab, and so on.

1
needpla•

144 drives. 8tb and 10tb. 3 24 bay netapp jbods. 1 super micro 72 bay 4u jbod. Everything is actually pretty quiet. Windows server.

Run out alphabet then map to folders hdd/1, hdd/2 and so on. My plot response time is never over a second.

1
CyberSKulls•

I got a few of the 72 bay SM chassis as well. Bought them on impulse cause I just had to have them. Nice dense machines

1
aj10017•

I have a 24 disk JBOD enclosure, no fancy RAID or anything, I just share a batch of 4-5 drives through SMB to my plotters until they are full, then I map the next batch of drives. I have a batch script that moves the plots from the buffer drive to the array semi-randomly to fill the drives equally, and it'll pause if there is no space left

1
dontbansme•

all bash scripts and cronjobs.. keeps it simple. basically scripts for formatting disks, finding drives with the most space, watching directories for plots, and then rsyncd. Oh also email alerts for duplicate plots or any errors. Def needed 10G fiber so I'm glad I upgraded last year

1