Laaaaaaaag

Servers are burning!

No, just kidding.
But we experience lags every now and then because of heavy CPU usage.
It seems that the hardware (mainly CPU I think) cannot handle the load sometimes.
Mostly is seems to be just fine even with every server running.
But sometimes the load seems to cummulate somehow.

Currently I pay 30 Euro per month for the server.
One solution would be to rent a stronger server.
But I dont want to spend more money for this hobby at the moment.
So if someone wants to donate for stronger hardware this could be an option.

Another solution would be to shutdown some servers.

Or maybe limit the number of players on each server.
Maybe 8 as maximum (this is anyway the number you see at the moment in steam even if 10 can connect)
Not sure if limit to 8 would really help much but it may be worth a try (without spending money^^)
Actually I prefer smaller teams anyway.
I never play on 16 slots servers and even on our 10 slot servers it is too crowdy for my feeling sometimes.
The team often is more like 2 or 3 teams playing in the same map.
Each team plays its own game^^
And L4D is also using much CPU, maybe I try to limit to 6 players there too (currently it is 7 but sometimes it produces more load as a full 10 slot L4D2 server).

Using vocal chat in the server also causes additional load.
Therefore I try to talk sense into the players that they dont play music etc over vocal chat.
So I could turn of vocal chat on the server.
But I don't know how much this would help really.

Another problem could be some plugin.
Maybe some plugins are not coded as optimal as they could be.
But it would be so much effort to test each plugin.

And the survival server sometimes produces much load too.
With 4 players the load is significantly higher as with 10 players on Coop.
Sure there are more hordes etc.

I promise to watch the loads and try to figure out some solution.
I will start with testing some of the mentioned solutions which don't cost additional money (limit to 8, turn off vocal, shutdown some servers, ...)

I think we don't need realism server no one plays there or plays not often

I know, but Realism is my favourite.
So one of them will stay for sure.

And an empty server only consumes some RAM.
The CPU has nothing to do if the server hibernates.

I thought about merging the second realism with survival realism.
So that you can change between realism and survival realism.

And maybe shutdown one L4D server but increase the players to 10.
So that the current 3*7=21 and 2*10=20 would be nearly equal.

My current feeling is that less servers with more players on each produces less CPU as more servers with fewer players.
But it is just a feeling at the moment.
I will have to test that.
But I only can test this with the servers being under full load.

Servers which will stay for sure:
1 L4D r0r
L4D classic
1 Realism
Coop Dyn
Coop
GunGame
Versus
Survival merged with Realism

With the merge and shutdown of one L4D we would have 8 instead of 10 servers.

If we close one of legacy server and increase to 8 players on legacy and clasic it will help a little and don't hurt people who play L4D.

GunGame stays, hooray! hehehe! ^^ 
I believe that the L4D servers must remain 3 (it's good to see that you just reduce the slots Albert!) cause these servers are full most of the time..many people love and continue to play L4D!  
Maybe it's right to merge the 2 Realism servers to one..i like Realism too but these servers are empty most of the times so i think that one of them will be just fine.
I wouldn't have a problem to donate a (small, haha! I'm unemployed right now, that's why "small"  ) amount of money every month for a better server if this will be necessary!

B.L wrote:

I think we don't need realism server no one plays there or plays not often

But I love realism, maybe we should keep at least one realism server.

What about clasic server i mean 2018 will be a clasic server and 2019 will be a server with plugins, then people can choose on wich server to play clasic or with plugins. I think it helps a little.

Hello,

L4D servers get full pretty fast since, i guess, some users cant run L4D2 on thei PC`s due to hardware limitations. I have to run L4D2 with low settings but i do run L4D with medium or high settings...

I wont suggest what server to shutdown or not, i am new here, but i can suggest lowering the number of slots.

I have seen russian L4D servers, with 12 - 16 slots, real crowded and sometimes no one to help you when down, campaigns fails due to unbalanced game play.

Later edit: as far as i see aLserv2019 Coop has 16 slots

Hey alBert, when did the laggs, or CPU started peaking? What if thats just a bad plugin that is very much resource hungry? maybe Tracking that plugin will be much more efficient rather than slot removing or HORRIFYINGLY REMOVING VOCAL CHAT, NOOOOOOOOOOOOOOOOO!

Or maybe its just a hardware issue? some of the servers componets just wore off?

Hard to say what is causing the issues.
Sometimes players complained while others on the same server had no problem.
Then its client related.
My feeling is, that the hardware is not strong enough to serve all 10 servers with many players on each.
99% of the day it is fine and there are even times a day where all servers are empty.
There a stronger hardware would be a total waste.

Plugins are definately a problem too.
Some are really coded badly and could be improved a lot.
But this is totaly time consuming.
And my feeling is, that a real bad plugin would make some server stand out a little from the others regarding CPU usage.
But I see the servers varying in CPU usage.

Hardware problem.
I hope not.
I hope this rented server will work for some additional months.
At least till I have a proper backup of everything^^

alBert2000 wrote:

...

Hardware problem.
I hope not.
I hope this rented server will work for some additional months.
At least till I have a proper backup of everything^^

A VPS is a container, in terms of virtualization that runs on a server.
Sometimes if there are other clients that use too much resources it could affect all the users, on the server or even worse the internet node (not sure if thats the case here).

Its good to have a backup of all the data from the game servers.

Well this is the reason why I have rented a real root server.
Not a virtual machine.

alBert2000 wrote:

Well this is the reason why I have rented a real root server.
Not a virtual machine.

Nice, its a good dead to get a real machine for 30 Euros. Does that include VAT?

Yes, it is actually 29 Euro in total.
See here:
[You must be registered and logged in to see this link.]
It is the "Root Server | Linux | Level 1"
It was the cheapest I could find with no additional costs like setup and it has a short term agreement of only 1 month.

We are taking about lag here and you increase to 16 on 2019 why?

B.L wrote:

We are taking about lag here and you increase to 16 on 2019 why?

Maybe they're testing whether or not the lags are caused by the SP or our client thingy..

Right. Testing.
To see how much more stress the slots make.
But my plan anyway goes to set them all to 8
I'm not a fan of many slots.
10 was too much.
8 seems to be a better number.

alBert2000 wrote:

Yes, it is actually 29 Euro in total.
See here:
[You must be registered and logged in to see this link.]
It is the "Root Server | Linux | Level 1"
It was the cheapest I could find with no additional costs like setup and it has a short term agreement of only 1 month.

That`s really nice. Well, hope to see more feedback in this topic, that way you know what decision/s to make.

alBert2000 wrote:

Right. Testing.
To see how much more stress the slots make.
But my plan anyway goes to set them all to 8
I'm not a fan of many slots.
10 was too much.
8 seems to be a better number.

I'm glad that it's only a test and 8 is the best.

Still deciding between 8 and 10.
Tested with 13/16 players today and it crashed on a normal map near the end.
So investigation is still ongoing (but not more than 10)
The balance would suffer with more players.
Everything was tuned to be dynamic till 10 players max.

Testing continues.
I saw many "laaaaag" messages in the chat.
Now I have turned off one L4D r0r server and one L4D2 Realism server.
The L4D servers have 8 slots now (instead of 6 before).
I'll test with this configuration at least for this weekend.
If you want to report something, check the time stamp of this post and report what you noticed after this configuration change.

I Will report.

I noticed the lag when a horde of zombies spawned, after you had stopped the 2 servers the load should be more or less lower on the server. Login in to the VPS, take a screenshoot of putty running the command top, another one when players start to join but servers still not full and of course another screeshoot when the servers are full, top command should output what process of the servers is RAM/CPU hungry than the rest (if the case)

On which server was the lag?
What time exactly?
How many players?
How was your ping (did you use "net_graph 3" to see your ping and network statistics)?
The lag when hords are coming is often caused by the client side because the PC is not strong enough.
Sometimes the map (custom maps) are also causing some lag effect (because FPS is dropping often). Is saw that on "Back to school" campaign yesterday. My PC was not strong enough to compute enough FPS all the time. Xeno saw the same. The ping to the server was fine, the "net_graph 3" did not show any exceptional in-game situation.

What does "VPS" stand for?
Mabye "Virtual Private Server"?
If "V" stands for anything "Virtual" than it may not apply because I don't run any virtualization. All game servers are running on bare metal (linux root server).

I'm using [You must be registered and logged in to see this link.] instead of top.

No need for screenshots htop is running all the time when I'm around the PC and one of my eyes is constantly watching^^
Further more Left4Maps2000 is running in parallel so that I can see the number of players on each server easily and if a map is changed at the moment.
All game server processes running on the root server are looking fine.
Non is consuming too much memory.
Sometimes a game server process is causing a CPU spike.
This happens when a map is changed an sometimes during a map too (in horde situtations? Player join does not seem to cause any problem.).
The spikes are short but maybe long enough to cause some lag on the other game server processes.
Maybe the huge amount of custom maps on each server is causing the spikes when maps are loaded? But I see CPU spikes also caused by L4D servers even if they have far less custom maps installed compared to L4D2.
Oh, and the survival is causing CPU spikes often too.
Maybe because of more hordes or maybe because the map is loaded there far more often compared to coop maps which take longer in general.
I think I'll have to investigate on the Survival a little too.
But I'm hardly never playing on that server^^

I have [You must be registered and logged in to see this link.] running on the root server to monitor everything all the time.
But it does not show the spikes because the granularity of measurement is not small enough I think (maybe I could modify something there for testing. Don't know. But this would cause more monitoring stress too.)
Here you see the CPU and memory usage of the last 24 hours:
CPU:
[You must be registered and logged in to see this link.]
Mem:
[You must be registered and logged in to see this link.]

On the CPU diagram of munin the spikes are not visible.
On htop I can see that the CPU is at 100% sometimes during spike times. (Cannot make a screenshot at the moment. Servers are empty^^. Only two servers are running on the following image).
htop:
[You must be registered and logged in to see this link.]

Investigation ongoing^^
I hope to see more heavy server usage this weekend to get more monitoring information.

Yes, my apologies, i forgot that it is a real server. I think you should stick with htop/top, since using a monitoring toll you end up using some resources for this program to monitor at his own, so stick with the linux tools IMHO.

I dont use net_graph when playing and i can`t remember the server name. All i remember is that the map/campaign was called Swamp Fever and i had played from around 22:30 to 23:00 Eastern European Time, GMagic was online too.

Maybe my PC is not that strong as well, setting graphics details on low may be good but stopping Vsync makes the image displayed... to be broken in parts when moving really fast in the game.

One paradox situation popped in to my mind: what if, the hordes happen to spawn at almost the same time on some or all the servers. No mather how much, just the event it self... can this happen ?