Centralized File Storage and Load Balancing

rated by 0 users
This post has 9 Replies | 7 Followers

Not Ranked
Posts 15
Points 225
P4 Healthcare Posted: Mon, Apr 28 2008 2:19 PM

Hi,
What is the recommended best practice for a load balanced environment and Centralized file Storage?

Currently, if someone on one of our servers uploads a file to CFS, half of our users are routed to a different server and will not see that users files.

I have seen some older posts on this site that recommend creating a UNC share, but that is really just a work around to defeat load balancing.  You end up with a dependancy on a single server to provide the full load for CFS and no redundancy.

Before CS 2008 removed SQL storage, we did not have this issue.  What is the solution for true redundancy of the CFS?  Is anybody using Robocopy or SyncToy to keep files syncronized in real time?  If so do you have a Robocopy script that you can share, which accounts for updates, deletes, etc?

Is anyone working on a SQL provider for CFS?

Thanks,
Graham

Not Ranked
Posts 8
Points 85
fredrik_r replied on Wed, Apr 30 2008 8:13 AM
This is a challenge for us too, and we are currently running live on only one server because of this. I would appreciate feedback on this issue. Regards, Fredrik
  • | Post Points: 5
Top 10 Contributor
Posts 2,423
Points 19,635
MVPs
Nick replied on Thu, May 1 2008 11:05 AM

You would need to copy the filestorage folder from the root of the site, and sync that between the servers.

I believe you could write a action for CS that would be run when a media gallery post is added.  This action would sync the two CFS's using the IO.  IIS would need to have permissions to do this however.

Doesn't windows server have an option to shaddow copy a directory?

CFS will not work directly in SQL, it's bad practice to store big files in a SQL db.

  • | Post Points: 20
Top 10 Contributor
Posts 5,292
Points 49,180
CS Developers

You could write a SQL provider, however it isn't recommended.

Typically, we will recommend using a UNC share.  That is usually sufficient unless you are doing high volume of uploading/downloading attachments.  The difference in performance is often negligible.

Another option you could consider would be using the Amazon S3 provider.

Anything that does syncing on the filesystem between two separate sets of files will leave you open.  IE, you make an upload, then a request comes in for the attachment before it is propagated to the other server, the user will encounter an error.

  • | Post Points: 35
Top 500 Contributor
Posts 75
Points 1,100

Just to know that I am reading this corrrectly.

With CS2008 you can specify a UNC path (network/SAN share - \\mySan\d\AllMyCSFiles) for the file locations since they are being moved out of the DB, correct?

I am hoping the answer is yes as this makes it much easier to manage. I don't even want to think about file replication and such especially as the community starts to go to 3,4,5 web servers.

Thanks,

Randy

thwack.com - SolarWinds

  • | Post Points: 20
Top 10 Contributor
Posts 3,424
Points 65,735
CS Developers

RandyRamirezATX:
With CS2008 you can specify a UNC path (network/SAN share - \\mySan\d\AllMyCSFiles) for the file locations since they are being moved out of the DB, correct?
 

Correct.

Ben Tiedt's Blog

  • | Post Points: 20
Not Ranked
Posts 7
Points 125
OnTheMike replied on Thu, Jun 26 2008 2:56 AM

Do you also have the option to login to the UNC-path under a different account (that's how we set it up)?

  • | Post Points: 20
Top 500 Contributor
Posts 100
Points 1,245
cpaul replied on Thu, Jun 26 2008 12:46 PM

On windows 2003/2008 server you could use the DFS (Distributed File System) to sync the directories.

For our site we can't have the traffic of the media interfering with the web servers, so we're basically writing our own Amazon S3 - like media server and CFS plugin (I'm sure that'll be fun).

http://www.curse.com

  • | Post Points: 5
Not Ranked
Posts 15
Points 240
snowcarver replied on Tue, Aug 12 2008 11:07 AM

Hi,

I've tried this setup and it works great for firefox and ie, however for safari running on a mac, not all the images show up.

In a non-load balanced environment where the filestorage is local, this problem does not occur?

Any ideas?

  • | Post Points: 20
Top 10 Contributor
Posts 3,424
Points 65,735
CS Developers
Ben Tiedt replied on Mon, Aug 25 2008 8:50 AM

Which technique did you try?  (1) a single file storage location referenced via UNC paths or (2) automated file copying between all web servers?

Have you installed CS2008 SP1?

Ben Tiedt's Blog

  • | Post Points: 5
Page 1 of 1 (10 items) | RSS
Powered by Community Server (Commercial Edition), by Telligent Systems

Copyright© 2008 Telligent Systems Inc. All rights reserved
CommunityServer.com  •  Telligent.com