[{"data":1,"prerenderedAt":472},["ShallowReactive",2],{"blog":3},[4],{"id":5,"title":6,"body":7,"description":462,"extension":463,"meta":464,"navigation":202,"path":468,"seo":469,"stem":470,"__hash__":471},"blog/blog/raid.md","RAID-ing things that shouldn't be RAID-ed",{"type":8,"value":9,"toc":456},"minimark",[10,14,22,27,37,50,53,69,72,76,79,85,89,92,442,446,452],[11,12,6],"h1",{"id":13},"raid-ing-things-that-shouldnt-be-raid-ed",[15,16,17],"p",{},[18,19],"img",{"alt":20,"src":21},"A ten-port USB hub full of flash drives","https://assets.quinngale.com/blog/raid/usb-raid-array.jpg",[23,24,26],"h2",{"id":25},"what-am-i-looking-at","What am I looking at?",[15,28,29,30,36],{},"First things first, there's a 10-port Sabrent USB 3.0 hub (",[31,32,33],"a",{"href":33,"rel":34},"https://a.co/d/6v8RTMt",[35],"nofollow",") strapped to the side of the computer I have running it. The hub gets pretty hot under load, so to hopefully fix that I have attached a fan to the back side of the case blowing air out of the case and over the hub.",[15,38,39,40,44,45,49],{},"The USB drives aren't anything special. They ",[41,42,43],"em",{},"were"," 10x 32GB SanDisk flash drives, like the one here: ",[31,46,47],{"href":47,"rel":48},"https://a.co/d/a1MCj78",[35],". Unfortunately, one of them stopped working while I was moving and putting this together so now there's just 9 of them.",[15,51,52],{},"The computer is my former desktop repurposed as a media, game, and file share server. For the nerds, the specs are:",[54,55,56,60,63,66],"ul",{},[57,58,59],"li",{},"Intel i5 7500",[57,61,62],{},"32GB of 3200MHz RAM",[57,64,65],{},"500GB Samsung SSD 970 EVO NVMe drive",[57,67,68],{},"Ubuntu Server 23.04",[15,70,71],{},"The drive is important because of how I set up the tests.",[23,73,75],{"id":74},"why","Why?",[15,77,78],{},"One reason:",[80,81,82],"ol",{},[57,83,84],{},"Why not?",[23,86,88],{"id":87},"methodology","Methodology",[15,90,91],{},"The goal of this is to benchmark USB drives in a series of different RAID configurations and see how they perform.",[80,93,94,126,157,185,261],{},[57,95,96,97,114,117,118,121,122,125],{},"We are using lvm to create logical volumes for RAID0, RAID4, RAID5, and RAID6, so we first need to set up the volume group with:",[98,99,104],"pre",{"className":100,"code":101,"language":102,"meta":103,"style":103},"language-sh shiki shiki-themes github-dark","# vgcreate usb /dev/sd{a..i}\n","sh","",[105,106,107],"code",{"__ignoreMap":103},[108,109,112],"span",{"class":110,"line":111},"line",1,[108,113,101],{},[115,116],"br",{},"The ",[105,119,120],{},"/dev/sd{a..i}"," will automatically get expanded to ",[105,123,124],{},"/dev/sda /dev/sdb /dev/sdc ... /dev/sdi"," on the execution side, and since our drives are all consecutive in our system we can do that.",[57,127,128,129],{},"Now we need to set up the logical volumes. I'm pretty sure performance stays constant through the entire flash memory block, so I am creating all of the RAID volumes at the same time. They are all going to be 16GB-ish due to the way LVM works.",[98,130,132],{"className":100,"code":131,"language":102,"meta":103,"style":103},"# lvcreate --type raid0 -L 16G usb -n raid0 -i 9\n# lvcreate --type raid4 -L 16G usb -n raid4 -i 8\n# lvcreate --type raid5 -L 16G usb -n raid5 -i 8\n# lvcreate --type raid6 -L 16G usb -n raid6 -i 7\n",[105,133,134,139,145,151],{"__ignoreMap":103},[108,135,136],{"class":110,"line":111},[108,137,138],{},"# lvcreate --type raid0 -L 16G usb -n raid0 -i 9\n",[108,140,142],{"class":110,"line":141},2,[108,143,144],{},"# lvcreate --type raid4 -L 16G usb -n raid4 -i 8\n",[108,146,148],{"class":110,"line":147},3,[108,149,150],{},"# lvcreate --type raid5 -L 16G usb -n raid5 -i 8\n",[108,152,154],{"class":110,"line":153},4,[108,155,156],{},"# lvcreate --type raid6 -L 16G usb -n raid6 -i 7\n",[57,158,159,160],{},"Apply a filesystem to each volume with:",[98,161,163],{"className":100,"code":162,"language":102,"meta":103,"style":103},"# mkfs.ext4 /dev/usb/raid0\n# mkfs.ext4 /dev/usb/raid4\n# mkfs.ext4 /dev/usb/raid5\n# mkfs.ext4 /dev/usb/raid6\n",[105,164,165,170,175,180],{"__ignoreMap":103},[108,166,167],{"class":110,"line":111},[108,168,169],{},"# mkfs.ext4 /dev/usb/raid0\n",[108,171,172],{"class":110,"line":141},[108,173,174],{},"# mkfs.ext4 /dev/usb/raid4\n",[108,176,177],{"class":110,"line":147},[108,178,179],{},"# mkfs.ext4 /dev/usb/raid5\n",[108,181,182],{"class":110,"line":153},[108,183,184],{},"# mkfs.ext4 /dev/usb/raid6\n",[57,186,187,188,232,234,235,238,239,258,260],{},"For testing, we are going to pre-compose 16 files, with sizes of 1KB, 10KB, 100KB, 1MB, 10MB, 100MB, 1GB, and 10GB and place them on the internal SSD. One half of them is going to be all zeros, and the other half is going to be full of random data.",[98,189,191],{"className":100,"code":190,"language":102,"meta":103,"style":103},"mkdir sources\n\nfor size in \"1k\" \"10k\" \"100k\" \"1M\" \"10M\" \"100M\" \"1G\"; do\n    for file in \"urandom\" \"zero\"; do\n        dd if=/dev/$file of=sources/$size-$file.dat bs=$size\n    done\ndone\n",[105,192,193,198,204,209,214,220,226],{"__ignoreMap":103},[108,194,195],{"class":110,"line":111},[108,196,197],{},"mkdir sources\n",[108,199,200],{"class":110,"line":141},[108,201,203],{"emptyLinePlaceholder":202},true,"\n",[108,205,206],{"class":110,"line":147},[108,207,208],{},"for size in \"1k\" \"10k\" \"100k\" \"1M\" \"10M\" \"100M\" \"1G\"; do\n",[108,210,211],{"class":110,"line":153},[108,212,213],{},"    for file in \"urandom\" \"zero\"; do\n",[108,215,217],{"class":110,"line":216},5,[108,218,219],{},"        dd if=/dev/$file of=sources/$size-$file.dat bs=$size\n",[108,221,223],{"class":110,"line":222},6,[108,224,225],{},"    done\n",[108,227,229],{"class":110,"line":228},7,[108,230,231],{},"done\n",[115,233],{},"The 10GB files had to be created specially due to some internal limitations in ",[105,236,237],{},"dd",":",[98,240,242],{"className":100,"code":241,"language":102,"meta":103,"style":103},"for file in \"urandom\" \"zero\"; do\n    dd if=/dev/$file of=sources/10G-$file.dat bs=1G count=10\ndone\n",[105,243,244,249,254],{"__ignoreMap":103},[108,245,246],{"class":110,"line":111},[108,247,248],{},"for file in \"urandom\" \"zero\"; do\n",[108,250,251],{"class":110,"line":141},[108,252,253],{},"    dd if=/dev/$file of=sources/10G-$file.dat bs=1G count=10\n",[108,255,256],{"class":110,"line":147},[108,257,231],{},[115,259],{},"I know there are more efficient ways to do this, but speed isn't important here and we have more than enough computing power to make it work.",[57,262,263,264,435,437,438,441],{},"Now, we have this messy bash file that essentially does a few things: It takes files that I've already created sized 1KB, 10KB, 100KB, 1MB, 10MB, 100MB, 1GB, and 10GB and times how long it takes to write that file to each logical volume and then how long it takes to read it back again. There's a 30 second pause between each segment to wait for temperatures to settle back down to normal. The results get thrown into two .csv files for analysis later.",[98,265,267],{"className":100,"code":266,"language":102,"meta":103,"style":103},"#!/bin/bash\n\ncount=\"1\"\nsleeptime=\"30\"\nsectorsize=\"16k\"\n\necho \"volume,source,trial number,file size,bytes,seconds\" | tee \"raid-results-write.csv\" \"raid-results-read.csv\"\n\nfor volume in \"raid0\" \"raid4\" \"raid5\" \"raid6\"; do\n    for sourcefile in \"zero\" \"urandom\"; do\n        for bs in \"1k\" \"10k\" \"100k\" \"1M\" \"10M\" \"100M\" \"1G\" \"10G\"; do\n            for rep in {1..20}; do\n                touch mounts/$volume/.raid_test\n                sleep 1\n\n                dd if=sources/$bs-$sourcefile.dat of=mounts/$volume/.raid_test bs=$sectorsize 2>&1 | awk 'END{print \"'$volume'\"\",\"\"'$sourcefile'\"\",\"\"'$rep'\"\",\"\"'$bs'\"\",\"$1\",\"$8}' | tee --append \"raid-results-write.csv\"\n\n                # echo \"Sleeping $sleeptime seconds for things to equalize before proceeding...\"\n                sleep $sleeptime\n\n                dd if=mounts/$volume/.raid_test of=/dev/null 2>&1 | awk 'END{print \"'$volume'\"\",\"\"'$sourcefile'\"\",\"\"'$rep'\"\",\"\"'$bs'\"\",\"$1\",\"$8}' | tee --append \"raid-results-read.csv\"\n\n                # echo \"Sleeping $sleeptime seconds for things to equalize before proceeding...\"\n                sleep $sleeptime\n\n                rm mounts/$volume/.raid_test\n                sleep 1\n            done\n        done\n    done\ndone\n",[105,268,269,274,278,283,288,293,297,302,307,313,319,325,331,337,343,348,354,359,365,371,376,382,387,392,397,402,408,413,419,425,430],{"__ignoreMap":103},[108,270,271],{"class":110,"line":111},[108,272,273],{},"#!/bin/bash\n",[108,275,276],{"class":110,"line":141},[108,277,203],{"emptyLinePlaceholder":202},[108,279,280],{"class":110,"line":147},[108,281,282],{},"count=\"1\"\n",[108,284,285],{"class":110,"line":153},[108,286,287],{},"sleeptime=\"30\"\n",[108,289,290],{"class":110,"line":216},[108,291,292],{},"sectorsize=\"16k\"\n",[108,294,295],{"class":110,"line":222},[108,296,203],{"emptyLinePlaceholder":202},[108,298,299],{"class":110,"line":228},[108,300,301],{},"echo \"volume,source,trial number,file size,bytes,seconds\" | tee \"raid-results-write.csv\" \"raid-results-read.csv\"\n",[108,303,305],{"class":110,"line":304},8,[108,306,203],{"emptyLinePlaceholder":202},[108,308,310],{"class":110,"line":309},9,[108,311,312],{},"for volume in \"raid0\" \"raid4\" \"raid5\" \"raid6\"; do\n",[108,314,316],{"class":110,"line":315},10,[108,317,318],{},"    for sourcefile in \"zero\" \"urandom\"; do\n",[108,320,322],{"class":110,"line":321},11,[108,323,324],{},"        for bs in \"1k\" \"10k\" \"100k\" \"1M\" \"10M\" \"100M\" \"1G\" \"10G\"; do\n",[108,326,328],{"class":110,"line":327},12,[108,329,330],{},"            for rep in {1..20}; do\n",[108,332,334],{"class":110,"line":333},13,[108,335,336],{},"                touch mounts/$volume/.raid_test\n",[108,338,340],{"class":110,"line":339},14,[108,341,342],{},"                sleep 1\n",[108,344,346],{"class":110,"line":345},15,[108,347,203],{"emptyLinePlaceholder":202},[108,349,351],{"class":110,"line":350},16,[108,352,353],{},"                dd if=sources/$bs-$sourcefile.dat of=mounts/$volume/.raid_test bs=$sectorsize 2>&1 | awk 'END{print \"'$volume'\"\",\"\"'$sourcefile'\"\",\"\"'$rep'\"\",\"\"'$bs'\"\",\"$1\",\"$8}' | tee --append \"raid-results-write.csv\"\n",[108,355,357],{"class":110,"line":356},17,[108,358,203],{"emptyLinePlaceholder":202},[108,360,362],{"class":110,"line":361},18,[108,363,364],{},"                # echo \"Sleeping $sleeptime seconds for things to equalize before proceeding...\"\n",[108,366,368],{"class":110,"line":367},19,[108,369,370],{},"                sleep $sleeptime\n",[108,372,374],{"class":110,"line":373},20,[108,375,203],{"emptyLinePlaceholder":202},[108,377,379],{"class":110,"line":378},21,[108,380,381],{},"                dd if=mounts/$volume/.raid_test of=/dev/null 2>&1 | awk 'END{print \"'$volume'\"\",\"\"'$sourcefile'\"\",\"\"'$rep'\"\",\"\"'$bs'\"\",\"$1\",\"$8}' | tee --append \"raid-results-read.csv\"\n",[108,383,385],{"class":110,"line":384},22,[108,386,203],{"emptyLinePlaceholder":202},[108,388,390],{"class":110,"line":389},23,[108,391,364],{},[108,393,395],{"class":110,"line":394},24,[108,396,370],{},[108,398,400],{"class":110,"line":399},25,[108,401,203],{"emptyLinePlaceholder":202},[108,403,405],{"class":110,"line":404},26,[108,406,407],{},"                rm mounts/$volume/.raid_test\n",[108,409,411],{"class":110,"line":410},27,[108,412,342],{},[108,414,416],{"class":110,"line":415},28,[108,417,418],{},"            done\n",[108,420,422],{"class":110,"line":421},29,[108,423,424],{},"        done\n",[108,426,428],{"class":110,"line":427},30,[108,429,225],{},[108,431,433],{"class":110,"line":432},31,[108,434,231],{},[115,436],{},"Just a copule of notes: ",[105,439,440],{},"$sectorsize"," is 16kb to match the default sector size that got set up with the logical volumes. And because the files are coming from an NVMe drive I'm not concerned about the write speeds on the USB drives outpacing the read speeds of the system drive.",[23,443,445],{"id":444},"results","Results",[15,447,448],{},[449,450,451],"strong",{},"to be filled in later once results are available.",[453,454,455],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":103,"searchDepth":141,"depth":141,"links":457},[458,459,460,461],{"id":25,"depth":141,"text":26},{"id":74,"depth":141,"text":75},{"id":87,"depth":141,"text":88},{"id":444,"depth":141,"text":445},"Results from putting 9 USB drives into various RAID configurations.","md",{"thumbnail":465,"alt_text":466,"draft":467},"https://assets.quinngale.com/blog/raid/usb-raid-array-thumbnail.jpg","An image of 9 USB drives plugged into a USB hub that has been strapped to the side of a computer",false,"/blog/raid",{"title":6,"description":462},"blog/raid","dJxrjKGGwRMuRF7meHoBMN-fY___xVkbDC4L641xl2o",1764741524554]