There is a problem I'm facing every time when uploading illustrations.
Problem is with binary files - slow to transfer, and they produce bloat. So it's not only slow and bloaty, but also slower and bloated more with time. Those are especially painful thing when using lightweight checkout as a local copy, with bazaar branch the speed is tolerable (for now), and the size is close to 1 GB, which is still not that bad. But it will be.
Also, version control is not designed to handle repositories that big, which consist of like 2% of text and 98% of bins.
Possible solutions:
dropbox, e-storage of any kind - it costs real money
ftp (-like) server - possibly free, easy, but needs some extra maintenance and planning to prevent vandalism and data loss.
using cvs that actually deals with binary files well, even if it's something really old and strange.
nothing else comes to my mind
Problems:
Providing artists with easy to use upload infrastructure
Providing devs with the same as above, plus some specs on how to use the files - like how to provide info on authorship, if it is better to link or embed the images in Inkscape, or how to (re-)structure directories when working with scribus files.
Also the problem is how to separate the content into what needs CVS and what not - for example I think Scribus files and Inkscape files vital for the project (templates, logos) could be versioned, even without linked assets.
Infrastructure for binary part of our project
I'm the filthy bastard you wish you never met.
Re: Infrastructure for binary part of our project
For simplicity I think dropbox-like services are the best option. It doesn't matter if they cost real money, I'll pay - they're not that expensive.
Here are some factoids;
dropbox
Supports 30-day or unlimited "version control":
Prices: 99 USD/year for 50 GB, or 138 USD/year for 50 GB and eternal version control ("pack rat"). Meaning it costs 2,76 per GB with eternal version control or 1,98 with 30-day limited version control.
There seems to exist clients for all major platforms, but they're not even needed to upload/download something.
ubuntu one
Seems to lack all kinds of version handling, but this is not confirmed yet.
Price: 30 USD/year per 20 GB, giving us a cost of 1,5 per GB stored data.
Clients exist for Linux & Win.
Here are some factoids;
dropbox
Supports 30-day or unlimited "version control":
See https://www.dropbox.com/help/11 for more info.Dropbox keeps snapshots of every saved change in your Dropbox folder over the last 30 days (or more with the Pack-Rat feature). So if your pet accidentally pressed the delete key and erased your memoirs or you simply saved a bad change, you can restore the file with a few clicks./../
If recovering files becomes a recurring issue, you can upgrade your account include the Pack-Rat feature. Doing so gives you unlimited snapshots of your files, allowing you to go recover any file as far back in time as you like
Prices: 99 USD/year for 50 GB, or 138 USD/year for 50 GB and eternal version control ("pack rat"). Meaning it costs 2,76 per GB with eternal version control or 1,98 with 30-day limited version control.
There seems to exist clients for all major platforms, but they're not even needed to upload/download something.
ubuntu one
Seems to lack all kinds of version handling, but this is not confirmed yet.
Price: 30 USD/year per 20 GB, giving us a cost of 1,5 per GB stored data.
Clients exist for Linux & Win.
Re: Infrastructure for binary part of our project
I'd rather avoid all the stuff tagged with U, even if it's opensource.
they give 2GB there, it's not bad, just same as dropbox, also with browser-only mode.
Also, we can increase the storage on our dropbox account slightly, registerring accounts with this link:
http://db.tt/UjBPzMm
they give 2GB there, it's not bad, just same as dropbox, also with browser-only mode.
Also, we can increase the storage on our dropbox account slightly, registerring accounts with this link:
http://db.tt/UjBPzMm
I'm the filthy bastard you wish you never met.
Re: Infrastructure for binary part of our project
Thumbs up for using 0.1 alpha! Just... don't count me in.
I'm the filthy bastard you wish you never met.
Re: Infrastructure for binary part of our project
AFTER checking out how expensive the hosting is, I think we can stick to the solution partially similar to what we have already - that is keeping incoming stuff on free dropbox account and storing/syncing, and (S-) FTP-ing bloat in a central place with lower availability (this may be even done in complete anarchy, like when syncing in direct p2p manner with p300 - which is not a good piece of software BTW, rsync or whatever with backup kept locally by everyone involved).
Also:
http://stackoverflow.com/questions/1044 ... r-binaries
This
http://code.google.com/p/boar/
is still in beta
Also:
http://stackoverflow.com/questions/1044 ... r-binaries
This
http://code.google.com/p/boar/
is still in beta
I'm the filthy bastard you wish you never met.
Re: Infrastructure for binary part of our project
I don't get it: How exactly does Board help us sync between us? From what I understood it's purely local...or am I missing out on something?
Sparkleshare has made plenty progress and they are contemplating going the same road as syncany with introducing hosting on FTP etc. The two projects seem to also be aware of each other, which is a good sign. I don't think we should use Sparkle unless it has other storage options than VCS:s.
I'm currently trying to get an understanding of how (un)stable syncany is, as I still thiknk it's the best - at current - stuff out there "on paper". Maybe should just run it in private on my machine for a while and see how it performs
Sparkleshare has made plenty progress and they are contemplating going the same road as syncany with introducing hosting on FTP etc. The two projects seem to also be aware of each other, which is a good sign. I don't think we should use Sparkle unless it has other storage options than VCS:s.
I'm currently trying to get an understanding of how (un)stable syncany is, as I still thiknk it's the best - at current - stuff out there "on paper". Maybe should just run it in private on my machine for a while and see how it performs
Re: Infrastructure for binary part of our project
Sir, you are just about to check alpha software, three months old. Doing this in the thread related to us looking for storage solution makes me think you will propose it to be used for production purposes, which I said already about.
In my everyday life, that is hobby and work, I struggle with software that needs more care than it got. I lose time, energy and nerves doing what is normally well-paid work, that is testing software and reporting bugs that are later simply ignored, or were "fixed" already in the way that does not fix anything (like when the text is supposed to wrap on the edge of a line, but it doesn't wrap, and the solution to that bug, that marks the bug as fixed, is putting manual line breaks... Sad but true reality.). There are some rare occasions when this is after all good - that is the case of using bleeding edge Scribus version, which gets lots of bugfixes and features we need, and not much stability loss (if any). But this is post-alpha software.
Now, I'm not saying that what you consider us to use is bad software, or bad developers. It's quite opposite: I understand what alpha means and I think devs are responsible, tagging the software with alpha.
Boar is VCS for bin files, not only local, but remote as well.
I'm not going to waste time testing things now. I (or "we", whoever that will be) will have to evaluate everything as soon as it will be at least in "usable beta", no matter how test would go now. For now the only right policy for me is to use software as it is, without relying on devs that may, or may not, push things forward in the future.
In my everyday life, that is hobby and work, I struggle with software that needs more care than it got. I lose time, energy and nerves doing what is normally well-paid work, that is testing software and reporting bugs that are later simply ignored, or were "fixed" already in the way that does not fix anything (like when the text is supposed to wrap on the edge of a line, but it doesn't wrap, and the solution to that bug, that marks the bug as fixed, is putting manual line breaks... Sad but true reality.). There are some rare occasions when this is after all good - that is the case of using bleeding edge Scribus version, which gets lots of bugfixes and features we need, and not much stability loss (if any). But this is post-alpha software.
Now, I'm not saying that what you consider us to use is bad software, or bad developers. It's quite opposite: I understand what alpha means and I think devs are responsible, tagging the software with alpha.
Boar is VCS for bin files, not only local, but remote as well.
I'm not going to waste time testing things now. I (or "we", whoever that will be) will have to evaluate everything as soon as it will be at least in "usable beta", no matter how test would go now. For now the only right policy for me is to use software as it is, without relying on devs that may, or may not, push things forward in the future.
I'm the filthy bastard you wish you never met.
Re: Infrastructure for binary part of our project
Q_x, you say that you aren't going to waste time testing things now, and that you are waiting for usable betas. For syncany this makes sense, but boar's website says this:
That sounds like 'usable beta' to me. I think you even pointed out the project. So you are testing this now, right?The project is in beta phase. There is no fancy GUI - only a command line client. It is very much usable, but not very polished.
Re: Infrastructure for binary part of our project
I agree that it's bad to rely on developers creating this and that solution that is supposedly a milestone and coming feature in a project, especially in open source projects where continuity and spent work hours aren't guaranteed.
Even if it is so we still are always in the mercy of the coders and it's also decided we prefer to use floss instead of commercial software if possible and not creating too much work flow problemas.
Yes, there is an immense difference between stable and proven projects that have been around and up and coming projects like Sparkleshare. While I personally believe that Sparkle and/or Syncany could become huge and well developed projects, it might take very long time for that to happen and for the code to be usable in a production environment. I'm aware of that, and it is of course nor reasonable for us to wait for e.g. 2 years for that to happen.
I do however not think it's very interesting how different dev crews describe the state of their software. If it is called alfa, beta or RC is the same to me - what I'm interested in is if it fill the correct functions for us and if it works as expected. Alfa x could be way crappier than Beta Y, while released Z could be shittiest of them all. In short, I think testing stuff is ok and more important than the labels, that, and general feedback from others ofc...
Even if it is so we still are always in the mercy of the coders and it's also decided we prefer to use floss instead of commercial software if possible and not creating too much work flow problemas.
Yes, there is an immense difference between stable and proven projects that have been around and up and coming projects like Sparkleshare. While I personally believe that Sparkle and/or Syncany could become huge and well developed projects, it might take very long time for that to happen and for the code to be usable in a production environment. I'm aware of that, and it is of course nor reasonable for us to wait for e.g. 2 years for that to happen.
I do however not think it's very interesting how different dev crews describe the state of their software. If it is called alfa, beta or RC is the same to me - what I'm interested in is if it fill the correct functions for us and if it works as expected. Alfa x could be way crappier than Beta Y, while released Z could be shittiest of them all. In short, I think testing stuff is ok and more important than the labels, that, and general feedback from others ofc...