This will be a storage server for servers not for use with laptop or pc for home use.
VMs will be linux servers running apps/services. NFS and iSCSI will be used by these VMs. Database servers will be iSCSI and the other non-performant servers will use NFS shares.
Lets say 10 VMs are the ones that really need fast block storage just like you expect spinning up EC2 with EBS volume in AWS. the servers using NFS will be similar to EC2 using EFS in AWS
Got it? I mean anything goes down even when critical with our lives depending on them. Doesnt your bank go offline every single day? Are bank websites critical? Even instagram, facebook that people are pegged to 247 can go down and for several hours too.
So you get the point. As long as backup and DR is setup, anything can be PRODUCTION. Next question will then be how often and how long downtime is acceptable.
Iām stumped.
You asked for help speccing the hardware and setting it up.
I asked you those questions to help gauge the usage case and you somehow managed to not answer any of them, instead being vague and lowkey talking about the definition of the word critical.
Maybe weāre just misunderstanding each other, but thatās okay, youāre under no obligation to share.
Instead Iāll go somewhere else while wishing you the best of luck.
1 Like
I did share. I mentioned this will be used by linux VM servers.
Sorry if you did not enjoy the figure of speech and exciting literature but i thought i answered your question.
Yes am here for help and yes am all ears to listen. So please do not get mistaken by how you interpret my responses
If you want specifics let me know, but like i said earlier this will be private infra setup so not some cloud setup or for running some webhosting service.
So will be a small operation thus 2U server and 24 drives in the first place.
This will ofcourse be used to scale later to bigger operation if lets say my SaaS app brings in millions of dollars per year.
This storage server will be in a datacenter to manage storage for servers running database, cache, web, app servers etc
SaaS apps running that depend on the storage device and there will be backup so someoneās livelihood which will be just me for now, does not mean disaster but just means couple of hours of trying to restore data and hoping not to lose too much transactions without the failure
Yes it will be on 24/7/365 and 366 sometimes
hope that helps
As a retired IT Senior Manager, I would say that there is a WORLD of difference between:
- a home lab;
- a small business internal use only (e.g. for a handful of users developing and testing a SaaS offering); and
- infrastructure supporting a SaaS service across the internet (all-be-it a small start-up SaaS).
I am fully with @neofusion here, because I would also be making completely different recommendations and configurations for each of these different use cases.
1 Like
more like this, i already mentioned this already in what i said
i already this will be used as storage for servers, and not laptops or PC
all of the other options you mentioned are used by laptop and PC for playing around
there is no playing around here and i also added that whether something is tagged with CRITICAL does not mean protection against issue, but rather one needs to have backup and DR plan in case any issue happens
you can setup the most critical setup and there can be fire at the location of the storage server, then how better was it than the home lab storage server then? that is just one of many examples of things that can happen, so dont just ignore that and say it is something out of control
One other thing to add, when providing recommendations for a server to store peoples data, always go for āshould not lose dataā except the person say hey i dont mind losing data for the fun of it. Then the person wouldnt have come here and lose data as that is not important. Just simple logic there. No one wants to lose data because its homelab or whatever is considered not critical. EVen a lab server should be treated as production/critical if at all plan is to use for production later on.
hope i answered this question yet again.
Want performance because of iSCSI? Hell Yes
Is this for homelab or testing and playing around? Hell No
Can you lose data and not be angry? Hell No
Will the storage be used for database servers that is transactional? Yes
What do you rank better? Performance vs Data Persistent: Data Persistent. I dont want to lose data. Will be used for iSCSI for database servers that should not lose data. Data integrity, persistence first before Performance. I rather have data than lose data because of focus on performance. That should usually be the choice other than a caching server.
TBH I can quite see why @neofusion got fed up - first you start by being cagey about your needs, then you drip feed it, then when we ask for a clear, unambiguous statement you start complaining about the quality of community support you are receiving for free from genuine experts (myself excluded) who are giving up their valuable time to try to help you.
Like @neofusion, Iām now out of here.
3 Likes
Alright i have spent some hours researching and so far here are the things am considering
I will turn off asynchronous writes so
Sync: ALWAYS
I dont care about manufactured write speed. All i want to gain from shared storage server is ability to have one place for storage for servers so i can consolidate the disks to one place and reduce duplication and wasted disks. Also need to be able to setup HA for some of the apps since VMs can come and go and storage stays ready to be consumed and mounted by new VM.
So that is kind of what am after. Yes write speed will not be great but i have data persistence, consistency and protection.
Eventually i will upgrade to all flash storage server when the money tree start to grow
Also regarding L2ARC, i really dont care much for this. ARC is fine as long as i can modify how much memory i want ARC to be able to use as this server will also be running a few VMs on them and i dont want ZFS to be hoarding memory in the name of read speed.
Nowadays everything stores things in memory and i rather let the in-memory data be on the client and not the server.
Databases have in-memory setup and that is fine so i dont care much for ARC either and i will even prefer to limit how much memory it can use, if possible
With this setup i dont need datacenter SSD with PLP as i dont care for ZIL or SLOG as i want Sync: ALWAYS. And for read speed i dont care for L2ARC either as i want very synchronous filesystem.
As i mentioned most client apps and services have in-memory capabilities so no need to be daisy chaining in-memory all across data path resulting in waste of memory really. And im-memory writes to me is recipe for disaster. I rather not have that ever even for a hobby project. Flash storage will fix need for that anyways. So no need to worry for too long.
For the setup of discs, my plan is to do a pool starting with 5 x 2-disk mirror vdevs for the iSCI pool. And then i can have other pools can just be a pool starting with 1 x 5-disk RAIDZ for vdev for NFS.
So now with all these comments, anything am missing or have no idea about? Please let me know about it
It is funny how people read forum posts and get offended for no reason.
Can we write expressively anymore? I mean do you live in a world where everyone is just passive and dance to your tunes all the time and the moment they disagree with you or have something to say back to you you call it quit?
How do you have conversations in life? Do you just cut people off because they got into a heated argument with you, and none of my posts are nothing near an heated argument if any argument at all
If you have a partner, do you just call it quit the moment they got something to say back at you?
Well it is funny how online forums can get when one is asking for help
Yes helping is not mandatory heck couple of benefits to helping. Google started free and look where they at. Many open source projects started free and some have scaled to big businesses
Anyways online forums are just a place to relax and loosen up. As long as someone is not directly attacking or calling you out of your names i think it is safe to enjoy it. No need to be overly sensitive to posts.
Respect the fact that peopleās time is valuable. Be concise, clear and directly answer the questions and youāll get the help you need. Being verbose, unclear and obtuse is not the way to get specific help from professional people with busy lives. I suggest you go back and in point-form re-answer every questions asked without duplication, assumptions or other vaguery and then you may get the answers you need.
1 Like
being reading docs, checking out posts and doing my own research. forums are just to help speed things up not that we are all noobs even though we may ask noob questions as a tactic to get down to certain details but when that doesnt work or becomes dramatic there is always option to just go another route
cheers
