Dawgclan Network

Welcome, Guest. Please login or register.
Did you miss your activation email?
December 17, 2018, 12:29:22 pm

Login with username, password and session length
Dawgclan Network New Forum

HomeHome HelpHelp SearchSearch LoginLogin RegisterRegister
  Show Posts
Pages: 1 [2] 3 4 ... 7
16  News / Network News / Status Update 25/11/09 on: November 25, 2009, 02:23:53 pm
Well I have decided it might be easier to just start a fresh Debian installation and migrate the few things we need that way it'll be cleaner.

I have Debian and MySQL moved to it, next will be the Radio IRC bot back-end and then fixing the MySQL replication.

The Radio is currently back online, however the request system is having some issues that I'll need to look into a bit later.

All in all though I am fairly impressed with the performance. I was able to make a Debian 64bit VMware image now that the new CPU is 64bit compatible and yeah overall performance all-round is good and a lot less noisy which is good.

EDIT:

Ok fixed a few permission issues on the back-end causing it problems writing to the database which was not allowing song queuing properly etc. It seems to have also fixed the Request page bugs.


All I need to do now is get the MySQL replication working so the website has the same database details as the back-end for track ID's etc which I'll try and work on either tonight or tomorrow hopefully have a few things going on which might cause some delays.

EDIT2:

Ok MySQL replication has been completed, things seem to be replicating well at the moment and requests are now working again.

I'll be re-structuring the Radio DNAS's a bit later and also still need to move the Radio IRC bot.
17  News / Network News / Status Update 24/11/09 on: November 24, 2009, 03:08:01 pm
Well the VMware Website was back to normal and I downloaded ESXi 4.

I have since installed ESXi on the new machine and that has gone fine.

I was able to convert our Linux machine over to a VMware image with little problems, it does require a bit of clean-up to remove VMware Server off it and also some of the HP management agents etc that are left over.

The harder part was trying to convert my physical disk image over to a VMware image. I guess it came back to bite me. We went with a Physical disk setup which was a bitch to setup to get better performance, however I don't think it should be too bad in the current setup.

I was able to ghost and convert the image over to an ESXi format so it's happy now.

I'll try and work on getting the VM's to talk to each other and clean-up the Linux VM this evening and see how we go.
18  News / Network News / Status Update 22/11/09 on: November 22, 2009, 01:24:37 pm
Well the radio will be down for a couple of days while I move to some different back-end hardware.

The noise of this old generation server is getting a bit much especially in summer, will be moving back to a Dual-Core AMD box which should be enough.

Going to move from VMware Server to ESXi I think to see how that works for us.

I'll have to convert our Debian OS into a VMware Image and the VMware Server image should just mount I believe.

Tried to download ESXi yesterday but VMware's website was in maintenance, hopefully they fix that soon and I can work on it.
19  News / Network News / Status Update 30/09/09 on: September 30, 2009, 03:08:45 pm
Finally got around to configuring my Secondary Slave DNS server. We now have redundant DNS.

Not that we needed it particularly for the Radio but it will be good for other things and future growth.

On the list of things to do are fix the Radio pages/MySQL replication that has broken since about a week or two ago due to running out of disk space for log files.

This took down MySQL and now that I have fixed the log in question to have a rotation of 1 week maximum this should hopefully not happen again.

I now need to check and fix the errors for the MySQL replication and it should start working again.

Also on the cards is fixing up the Radio DNAS's. I will be re-arranging this in the near future.
20  News / Network News / Happy 4/20 Everyone on: April 19, 2009, 11:23:13 pm
Hope you all have a good one.
21  News / Network News / Status Update (29/03/09) on: March 28, 2009, 11:20:56 pm
Well looks like everything is falling into place now.

I have just finished the MySQL replication after re-compiling with SSL support. So now the requests and other functions are working again.

I need to change a few of the website scripts to be more dynamic but it all works as of current.

I guess I'll work on moving the other DNAS cluster soon.

Then the FTP Upload server and then the memory leak in our back-end Eggdrop script.
22  News / Network News / Status Update (28/03/09) on: March 28, 2009, 04:50:56 am
Well started messing around with VMware Server 1.x and was running into all kinds of issues again.

I finally got it compiled though and the raw disk support working. I was then able to start fresh with a new partition structure and installation and get Windows working.

Even though this was working I was still receiving some errors with the real-time clock and a few other things but they didn't seem to stop the VMware from working.

I then tried streaming the audio but found it was skipping/buffering for some reason on the client-end however the server wasn't throwing any errors and didn't seem to be having problems buffering?

The unfortunate part is that VMware Server 1.x is out-dated these days and was built for older Linux kernel's. A number of core functions it relies on have since been modified causing warning/errors during compiling. There are all sorts of hack-like patch files people have made to try and change the include files and modify functions which will get it working but it's not performing 100%. our compile was using 2-3 patches just to get the setup working.

I tried a number of things including recompiling the kernel a number of times to try and change the RTC device driver and did not have any success still had buffer issues.

So I said hell I'll try VMware Server 2.x one more time since we have 1.x working again on the new kernel.

I don't like VMware 2.x as much as a product because of their removal of the Server Browser and the introduction of their ESX style web interface administration and Infrastructure client. These to me have benefits but was not as simple/easy to use as just having a single executable that did its job like the browser did.

That being said I installed VMware Server 2.x and being a newer product it has some fixes for most of the newer kernel include file changes. This means it can compile without patches. I was able to apply one patch to get everything working for us and VMware Server 2.x booted! It also did not have the buffer problems that VMware Server 1.x seemed to have and did not throw any RTC errors.

So we were finally able to get VMware Server 2.x working. The good news too is that the guest as mentioned is performing a lot better in VMware than it was with VirtualBox for what we need to do. It has a consistent cpu load and seems a lot smoother under load. In VirtualBox the host cpu usage was spiking sporadically and kicking the server fans into high gear.


So finally we have success and I can now go back to what I was doing which was fixing the MySQL replication for requests etc.
23  News / Network News / Status update (25/03/09) on: March 25, 2009, 04:05:54 pm
Alright.

Well I tried Virtualbox, it seemed to be going good in the beginning. VirtualBox in the beginning was it was outperforming VMware Server 1.X however on closer inspection I think in heavy CPU load environments it doesn't cope as well.

One of my disk-image based vmdk's worked fine which ran Linux. I tried to boot my raw disk though and it would say "OS Not found" I realize I would probably have to repair install but I figured it would at least boot the OS and blue screen.

So I ran a repair install which didn't work. I then went through with a fresh install using the same partition and installation over-writing it, this also did not work...

I had to remove the partitions and start again to get up and running, not sure why the raw disk of VMware vs. raw disk of VirtualBox would have slightly different partition table structures?

In any case I got the OS loaded and started running some applications, the host CPU usage was hitting 100% for the application and the fans in the server were going crazy.

I did a lot of reading around including the "HAL" changes to Single processor, they did not help. One I did read from way back was loading a second virtual machine makes VirtualBox behave. For the hell of it I loaded up the second VirtualBox and paused it and my CPU usage dropped under real load to around 30-40%

So loading the second VirtualBox instance for some reason made the CPU usage normal? Hopefully this gets fixed soon.


On to my findings anyway:
Under no load the Virtualbox seems fairly fast, responsive and I like the fact that using VRDP you can achieve better resolutions than with VMware relying on the host graphic card or similar?

However I found that opening programs caused bigger CPU spikes than VMware Server seemed to on the Host and when you start to put the guest under load it in my opinion starts under-performing VMware Server. When there is a lot of intensive CPU operations happening in the guest things start to slow down a lot where as in VMware Server it seemed usable.

Conclusion:

I thought I'd post my findings of where things need fixing and improvements and how I found VirtualBox stacking up against VMware Server. I do look forward to seeing how VirtualBox progresses though and it seems to run quite well in most things.


So now I need to look back at VMware Server 1.X fun stuff.
24  News / Network News / Status Update (22/03/09) on: March 22, 2009, 02:32:30 pm
Well that was a fun weekend, well not really.

I started working on the MySQL replication and was doing pretty good. I have almost got that finished, just need to be recompiled with SSL support I believe.

In the process I decided to upgrade the main back-end server to Debian Lenny previously running Etch. So the versions of MySQL etc were hopefully the same.

The upgrade involved of course changing the kernel to the latest stable Debian version 2.6.26 and upgrading most of the packages to the latest builds.

In the process of this it broke our previously running VMware Server 1.0.4 as the kernel modules were removed (outdated).

I spent most of the weekend trying to get VMware Server to recompile on the new kernel...

I started by downloading the latest version from VMware's site which I thought was 1.0.8. VMware Server 1.X doesn't like to build on the new kernel and reading around there are some dodgy hack work-arounds that attempt to "patch" VMware files to work on the newer kernels because some of the source include files have changed kernel-wise.

These don't easily work. I did at some points get the VMware Server 1.x installed but the modules were either mis-matched versions because of the hacked up patches (injecting wrong version numbers or for products most likely like VMware Workstation or similar) which I figured this is bound to crash later or it would install but not want to load our previously working VMware images...

Now we use a "raw disk" which VMware class as an "Advanced" setup, though it seemed relatively easy to setup after you got around VMware's stupid shit wanted a /dev/sdX device for example instead of our pure RAID device /dev/cciss/cXdX or similar. However it is changable later in the config files... anyway enough about that back to the story.

"raw disk"means rather than using a VMware image file we actually talk to a physical RAID array (In our case) or hard-drive. It did not like to talk to it after the Kernel upgrade for some reason, I tried a lot of things.

I then found the new VMware Server 2.0 on the website I had completely overlooked. This compile against the later kernels without any hack, however it throws a few warnings along the way. The funny part is VMware decided in 2.x that they would remove "raw disk" support in this version...

Now VMware Server 2.X is a bit of a jump from 1.X they are trying to what it looks like make the look and feel feel like their Enterprise products like ESX. They now have a "Web Interface" over the previously great and usable "VMware Server Console". Then yeah they decided to remove some great features (supposedly because they were either strapped for time or believe they were not high on the priority list which sounds like bullshit) like "raw disk" support, now reading around supposedly the code is still there but I could not successfully get it to load the old VMware images. I tried a lot of things and spent a lot of time reading.

In the end it sounds almost like if you have a standard VMware image and then you can add a physical "raw disk" to the image which would be usable in the guest. I needed to boot off one and it would just not work...

So after spending a day and a half trying to get VMware I finally decided fuck VMware.

I looked at Xen Source which I didn't realize was Open Source again since they were I believe bought by Citrix maybe? however it seems it is and these days has Windows guest support. The unfortunate part is after reading around the Windows guests are only usable in Xen if you have "hardware virtualization" VT. So that threw my Xen ideas out the window as the server we are currently using is an older generation.

I then saw VMware ESXi is "Free" however it's a platform OS, I didn't really want to remove the entire operating system just to then run 2 virtual machines instead of 1? We aren't making massive Virtual Server machine.

So finally I looked at Sun's new VirtualBox, I know people who have been using it on Windows and I knew it would probably work on Linux however I didn't know how well it would run and whether it could start automatically on system boot, require user intervention etc...

So I read the entire manual for VirtualBox and it seems they have a tool included which is VBoxHeadless which allows you to run a VMware machine without X11 and it just spawns an instance with remote RDP access.

Brilliant. Now originally I originally thought "RDP" well that obviously means it's fairly limited I can't view DOS/console when the system is booting, what am I going to do for Linux if I wanted to use it" etc. It turns out they are talking about a "VRDP" (VirtualBox RDP) which is actually their own sort of protocol (built using standards compatible with RDP) and allows you to see the console and boot-up etc.

Perfect, so I went about loading up one of my older VMware test images in VirtualBox and it actually worked quite well. It loaded first time, I had to obviously load the Guest Tools but I was able to connect using RDP from my Windows machine and was able to get it all working. In some ways it actually feels more responsive and you can use higher resolutions, however we are not currently using all the processing power of our Radio Virtual machine which could slow things down.

Now I still don't know how the performance of VirtualBox will be in comparison to VMware when doing heavy streaming/encoding, so at this point I still don't know how it will perform.

I have checked the manual as I said and VirtualBox does allow "raw disk" support.

So hopefully later on I will try and mount this up, will most likely have to repair the Windows install as the Virtual drivers are probably different and then see how we go.

It looks good so far though and VirtualBox so far has been leaps and bounds easier than VMware (Once you understand their systems) and hasn't had issues on the more recent Kernels like VMware Server was.

I understand VMware Server is "free" and their focus is more on their enterprise products, but fucking up VMware Server like they have with an ESX style web-interface which is absolutely shit, sometimes gets stuck causes because of most likely poor Javascript/AJAX which can sometimes cause VMware to crash and then removing great features like "raw disk" support which they originally had is fucking frustrating. you can use a "VMware Infrastructure Client" to connect to VMware Server 2 however it's limited in the fact you can't import VMware configurations for example only create new ones.

In any case hopefully I can get some Sun-shine from VirtualBox and get everything operational again and then finally fix the MySQL replication. That will get two things out of the way...
25  News / Network News / Status Update (15/03/09) on: March 15, 2009, 02:26:22 pm
Hi All,

just a bit of information about the recent turbulence in regards to the website being offline at times, the servers being offline and requests not working etc.

We are moving a few of our servers around, one of our Shoutcast DNAS cluster boxes was recently re-imaged because it had a Beta Linux Kernel which became bugged. It has now been re-imaged to the latest stable release, as such a lot of services had to be re-configured/setup so I decided I'd take the time to create a Jail/Chroot environment for all the services.

We have now moved the IRC server and DNAS cluster into this environment, also one of the public Eggdrop configs.

I have still yet to install the FTP upload server. That will come hopefully soon.


In regards to the website etc, it has been moved to a new server. This box is more powerful than the old one and also has redundant hard-drives (RAID1).

I believe it was only a matter of time before the single drive in our old server failed and since ThePlanet did not want to give us a break after being with them for a number of years and would only offer an upgrade for $40/m for the RAID card and probably another $30-$40/m for an additional disk I thought it was time to move on.

So for a short period (or maybe not so short) the website and a few things like forums/scripts that were hard-coded or had hard-coded configurations broke because the paths have changed slightly. If you find any "issues" let me know.

As the weeks progress I hope to have more of the services migrated over to the new box.

At the moment a few things still need to be migrated, like another one of the DNAS clusters and the MySQL replication needs to be rebuilt for the new servers so that it has a local copy to handle requests etc.

Once the major parts are done, I'll look at the rest.


Also I am interested in getting the website re-done, as many of you know the website has been in a state of "half-design" for a long time now, I think it's better to just look at a solution like Joomla, so if you have any design skills let me know. Hopefully then we can get more activity.

Once all this is completed I will probably finally look at a memory leak in our back-end Eggdrop script which seems to slowly eat all my RAM on the server, need to narrow down what is causing it as there is a fair bit the script does.
26  News / Network News / Status Update (05/12/08) on: December 05, 2008, 10:34:42 pm
Well we are still around, though not very active currently.

It's that time of year again, X-mas and as such have added the X-mas songs back into song rotation for the season.

Also cleaned up the forums of the spam that seemed to surge the last couple of months.

If you have any feedback post on the forums.
27  News / Network News / Status Update (24/02/08) on: February 24, 2008, 02:40:47 am
Well I have finished the Genre query and Stream Title script these are now reported in the IRC Bot which makes it pretty much complete.

I also created the status query script and it looks to be working, there was a brief outage of the 32Kbit streams while this was tested. So now the bot and the Radio page should reflect whether the station is really up or down.

The next thing I'll work on is the Anti-Stream ripper script. Hopefully I can start this soon.

On another note I am in need of a Linux/Unix programmer, if you have any skills in this profession please get in contact with me, this will be for a project to allow for live DJ's and simplify my Hub setup and in turn the IRC Bot also. I need someone fluent in Linux/Unix programming though.

That's all for now.
28  News / Network News / Status Update (21/02/08) on: February 21, 2008, 04:41:34 pm
Just a quick update to report a fix on Forum registration, the image was not showing up, this has now been resolved.

I have also started work on the Genre and server title query script which is about 80% complete, I should hopefully have this done on the weekend maybe. Then I can move onto the anti-stream ripper script.
29  News / Network News / Status Update (20/02/08) on: February 20, 2008, 01:05:17 pm
Server went down for a while, this was due to the fact it lost network connectivity, much like we were having on the other server...

I don't know what causes this as this is a fresh install must be a bug in VMware. After rebooting the VMware it still did not have access to network resources, went to reboot the server and MySQL hung. This has also happened a few times, very strange might try and upgrade MySQL soon to see if it resolves that issue.

On another note, the missing module that was causing skipping of tracks has now been checked and auto-loads when the server boots so that is good and I took this time to resolve a few issues with the Mp3 share authentication failing on boot of VMware. So everything is slowly getting better.

I will be looking into getting a status check script on the IRC bot and also see if I can't reflect these kind of issues in the MySQL database (so it updates the website pages etc), I'll probably query the hub servers to see if they have a connected incoming source.

I'll see how I go with that.
30  News / Network News / Status Update (17/02/08) on: February 17, 2008, 06:09:51 am
Well I have finally got around to finishing the RSS feed again

http://www.dawgclan.net/rss/index.php

Please give any feedback you have, I was thinking about possibly making the "Currently Playing" and "Recently Played" descriptions into separate items? I am not sure how other peoples RSS feed readers look at the data though... Let me know how it looks in your RSS reader.

On another note we have another sponsor back, used to be SHOUTcastHosters, so thanks goes out to the new Logical Host.
Pages: 1 [2] 3 4 ... 7
Powered by MySQL Powered by PHP Powered by SMF 1.1.7 | SMF © 2006-2008, Simple Machines LLC
SMFone design by A.M.A, ported to SMF 1.1 RC3 by Aäron.
Valid XHTML 1.0! Valid CSS!
Page created in 0.026 seconds with 18 queries.