I've been asked to store media files in the database instead of on the file system. The one advantage to this is not having to use the UNC Path overrides. The disadvantage I see is, well, I have to write more code. Another potential downside is if there is a HUGE number of media files that could be placed into the db; this shouldn't be as much of a concern because it will be used internally at a company of <100 people.
So, I've been looking through some documentation to try to figure out the best way to do this. I read this page and figured the best way to go would be to, first off, implement a class deriving from CentralizedFileStorageProvider.
I had a couple of questions on how to proceed.
Thanks for your help.
-Glen
The idea with the path parameter is that it is a unique resource locator. So, in the database implementation this could be your unique identifier - whatever you want that to mean. A Guid would work just fine. When files are stored via your provider you get to makeup what this path will be. You just need to be able to take the path back in from a request and return the correct resource.
For caching purposes I could also see running things like legacy CS, and placing files on disk for immediate retrieval and only going to the db when not on disk. If you do this you should take the lesson from the file-based cfs and store files in sub-directories. Windows doesn't handle having 10s of thousands of files in one directory very well.
I figured out what was happening that was causing me such great confusion. The "path" is just created each time from the PostID that is stored with the PostAttachment. If a GUID is actually passed into the FileStorageProvider, then it is a temporary file (and will be deleted). So, I'm just doing a check on the FileStoreKey. I can either use the GUID that is being passed in as the MediaId in my database, or create a new GUID and use the path that is passed in as the PostId to store in with my media file in the database.
I actually need to do this so I can retrieve the file later because there is no reference in the cs_PostAttachments table to the GUID stored in my media database. Therefore, when I request to delete a file, it has to use the PostId as the reference to find and remove the correct entry in the DB.
I haven't completed everything yet. I still need to work on displaying the media once it has been stored.
If I get all of this taken care of, later, I will work on storing the files in the file system as well. I am thinking about just storing it in the local ~/filestorage and checking there first when the file is asked for. If it is there, it returns it, if not, it will go to the DB and create a copy of the file on the file system, and then use the file system's version. If I'm feeling really ambitious, I might set up a task that runs every day or every week that clears the ~/filestorage of files that have not been accessed in the previous week (or some other given amount of time).
Creating your own CFS provider is correct. The only way to know if you should call base is to look at the code for base unfortunately. There is no hard and fast rule for this when extending a class.
Depending on the size of the database, it could make sense to keep it separate if you will want to manage the backup schedule independently. There really isn't any harm in keeping it all in one if you have the files for your CS database managed properly as it grows - place your table in its own file set.
Thanks for the info Jeff.
I'm just now getting around to doing this.
I noticed that there is a "Content" column in the cs_PostAttachments table; I assume that is just something that was left over from previous versions. I wonder, though, if it would be easier to just use this table than to create a new table outside of the current CS database.
In the CentralizedFileStorageProvider implementation, what should be passed around as the "path" parameter? Since the files will be in the db, I'm just not sure if that is necessary.
Thanks
Thanks Jeff, that got me started pretty well.
I've implemented my "DatabaseFileStorageProvider" and it is storing the file content to a separate database like I expect. However, I am having problems with the path still.
One problem I'm having is that the file is still stored on the file system, but instead of a path like "00\00\00\23", it uses the GUID that I created to be the path. So, it splits up the GUID, creates the directory structure from that, and writes the file to the file system.
The other problem is that when the GetFile method is called, the path passed in is always of the form "00.00.00.23", instead of the expected GUID. Of course, it never finds the file because I have an extension method that tries to convert the path into a GUID. If it can't, it throws an error.
I suspect the first problem is somewhere in the SqlCommonDataProvider, but I can't pinpoint it, other than the MakePath method there. I'm still uncertain of where the second problem might be happening, would it also be something in the SqlCommonDataProvider class?
Any further direction would be greatly appreciate.
I guess if I were doing it I would use 3 things as a key (not necessarily a pk in the db, but as unique for retrieval)
1. The store key
2. The path
3. The name
These 3 things allow you to locate unique files of common names both within and across storage types. The path you are given is a reasonable storage identifier for the store type (userid, postid, etc).
The problem I'm running in to with the path, though, is knowing what the path that is passed in by CS means. For instance, when looking for user files, CS passes in the user's ID (say 2100). So, the path is 00.00.00.21.00 (or something like that). I guess I just have to catch the path, and then figure out what it is supposed to mean by checking the FileStoreKey.
I haven't seen any documentation on what form these paths from CS might take though, so I guess it will be a game of trial and error.
Something else I just found out is that the MultipleUploadFileManager is going to my Media database and deleting things that are "expired". So, now I have to figure out how to make it appear that these things are NOT expired.
Thanks for your help and input, Jeff.
The path is really just a unique identifier - it doens't have to "mean" anything. Taking avatars for example, any avatar file stored by user 2100 will be in that particular folder if you think in terms of file system. This makes all of user 2100's entries go to one place because the provider's actions are unique to a particular key as well - CommunityServer.Components.Avatars in this case. Finally, the name of the file is unique within that sub-division. You can see that the avatar storage component just makes up this path based on the user id:
private static string MakeCfsPath(int userID){ string idString = userID.ToString(); idString = new String('0', 10 - idString.Length) + idString;
return CentralizedFileStorageProvider.MakePath( idString.Substring(0, 2), idString.Substring(2, 2), idString.Substring(4, 2), idString.Substring(6, 2), idString.Substring(8) );}
Other types of stored files use the postId, or just a Guid. The static MakePath method on the base provider just joins the components of the params array into a dot delimited string.
Copyright© 2008 Telligent Systems Inc. All rights reserved CommunityServer.com • Telligent.com