I think I’m ready to elevate from bonded 1G networking to 10GB. I tend to have used/old netgear equipment at home, purchased on the cheap. I think I’m finally seeing enough latency when trying to run OS’s on my proxmox server with storage on my TrueNAS that I want to move to 10G networking.
Looking to replace my existing 12-port L3 switch (6 ports are occupied) with either RJ-45 or SFP based 10G, I think I will need realistically 16 ports. Storage will specifically be moved from my 48 port L2 witch with 4-1G bonds, carrying storage and networking, to probably 2 10G bonds dedicated to storage, which currently means I need an additional 6 ports right now, so 12 total used, then planning for future expansion, 16 port minimum.
I’ve been crawling ebay, but the barrier to entry is way higher than my foray from unmanaged to managed networking. I keep landing on Netgear M4300, where the price ranges from about $1000-$2400. I can save up for these, but is there a better way to get into 10G? I’ve seen some HP Procurve switches that seem way more cost reasonable, but my networking peers have told me to completely stay away from HP networking.
Any thoughts are appreciated!
Check out the FS stuff. It’s pretty cheap brand new
https://www.fs.com/c/10g-enterprise-switches-3256
They SFPs are cheap and work very well too.
This. FS or there’s some shockingly good (looking) stuff on Aliexpress I’ve been considering and have heard good (perhaps sponsored) things about.
If you wanna dip your toe in the 10Gbps water, I recommend you giving Mikrotik a look, specially CRS305-1G-4S+IN or CRS309-1G-8S+IN. The former has one 1Gbps RJ45 port plus four 10Gbps SFP+ cages while the latter has 8 10Gbps SFP+ cages.
Mikrotik stuff is pretty good and quite inexpensive. They all run RouterOS, which is their linux distro for switches/routers. Fully managed and can do basically anything L2 or L3 you could want. Worth noting that availability and pricing has been pretty variable from their resllers for a while, and you will pay a premium via Amazon. Here’s a list of relevant product product pages with the port counts and MSRPs:
- A router with 12x SFP+ and 2x SFP-28/25g for $600, I run one of these as my router.
- A switch with 16x SFP+ for $500 Probably the right switch for you
- A switch with 24x SFP+ and 2xQSFP/40g for $600 I run one of these as my core switch
Also these may help for some corner cases or change your ideas on what you want to build:
- A tiny switch with 4x SFP+ for $150, I have a few of these where I want multiple things plugged in to 10g, but don’t want to run a cable for each of them as the devices aren’t usually using a full 10g at the same time.
- A switch with 8x SFP-28/25g and 2x QSFP-28/100g for $1000, interesting because each QSFP-28 can be broken out into 4x SFP-28, so a total of 16 25g ports. Just imagine making the jump to 25g instead of 10g, wouldn’t it be glorious?
- A switch with 48x 1GBASE-T, 4x SFP+, and 2x QSFP+ for $600 Who needs anything else? Break-out the QSFPs into 8 SFP+ ports, you’ve got 4 SFP+ ports natively, and then another 48 ports of 1g copper. Could be pretty interesting. There is also a variant with PoE output for $400 more in case you need to run cameras/APs/whatever and don’t want to run injectors.
- A super weird 2x SFP-28/25g network card that is actually a full router for $200 I haven’t figured out a use for these yet, but I kinda want one. Otherwise my SFP-28 ports are being wasted at just 10g, right?
- A switch with 16x SFP-28/25g and 2x QSFP-28/100g for $1600 Total overkill, but I just had to list it since it falls in the middle of the price range you were considering but is so much beefier than the Netgear offering.
The Brocade ICX switches are very affordable for the performance. Great way to get a mix of 1G PoE, 10G, and 40G for under $200. Sadly they do consume a fair bit of power and most are loud out of the box. Although, there is a fan mod you can do to make them run quiet and cool.
Here is a link to more info: https://forums.servethehome.com/index.php?threads/brocade-icx-series-cheap-powerful-10gbe-40gbe-switching.21107/
I have an actual bedroom closet with a 45U rack and dedicated power and cooling, so noise/power are not a concern, I’ll look into it. How is the mgmt of them?
Management is mostly done through CLI. You need a generic console cable to get it on the network and then you can do everything remotely. There is a web ui as well, I mostly use it for quick status checks.
I have only used an ICX6610-48P-E. As this was my first non-Unifi managed switch, I spent a few hours setting it all up. Now, I just go back to the guides on ServeTheHome whenever I want to change VLANs or something complicated.
My home server is connected to the switch using a Mellanox ConnectX-3 and a QSFP DAC cable. It was about the same price as a 10G card and link so I figured why not go 40G to future proof it.
