Page 1 of 1
The Future of Terrasync ...
Posted: Tue May 03, 2016 11:20 pm
by IAHM-COL
...spells death
I had been reading up Torsten new ideas about terrasync, and if I get this right he is killing SVN.
So....
The scenery (OFFICIAL) will have no development sandbox of any kind. Keep updated means downloading the whole 100GB again via http. terramaster will stop being functional.
TerraGIT seems, to me, the only viable function that will be left. And that is -- I must say--- in part, thanks to it not being part of the core developers at the mercy of Torsten not needing to question anyone about his ideas.
Outrageous.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 12:19 am
by jwocky
So, well, we can't always stop them from killing themselves, all we can do is keeping stuff alive on our own ways.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 1:40 am
by legoboyvdlp
No!
I have a 100 KB/S internet -- I cannot download 100GB.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 1:45 am
by legoboyvdlp
It looks bad. I will test, and if it is as bad as I thought, I'll switch to TerraGit. I was too scared before, due to tile sizes and internet.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 3:17 am
by legoboyvdlp
I should let Torsten answer, but he's likely fast asleep right now (Europe), unless he has to get up to let out the dog ... in which case I would recommend he stays away from his computer so he can get back to sleep.
My understanding is that the http download method will function very much as before where only the changed files (if any) are downloaded.
The svn method requires 2x the actual space because of the way svn works under the hood. Git typically requires that you download every version ever created to get the latest copy (there are work arounds if you just want the most recent version.)
So the end result for the average user that just wants to download and use the scenery is that the http method will be lighter weight overall. It also simplifies the code inside FlightGear quite a bit and should make the synchronization more robust. And as mentioned before, it should simplify the task of setting up a scenery mirror for those that wish to do that.
As before, power users will have multiple options for updating scenery or mixing and matching from different sources if they choose.
Best regards,
Curt.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 6:48 am
by J Maverick 16
I love TerraSync, I can't live without it...
Why they keep destroying things that mostly all the FG pilots use?
Regards, Mav
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 2:29 pm
by IAHM-COL
Torsten in the Devel.List wrote:Let me try to calm down emotions here.
The technique of the http repository is much simpler than that of the svn repository and it will be pretty simple to create a local mirror or to download chunks of data.
With a bit of to-be-developed scripting (python e.g.) there is no need to download existing data again. I don't know how terramaster works but I am sure it is easy to adapt to the new http access method.
I apologize to not have everything ready at once and only implement the transition step-by-step.
The first step was to provide the necessary infrastructure. This is complete, we have one public master hosted at sourceforge and to privately donated mirrors.
The second step was to implement the client to access the infrastructure. This is complete, thanks to James for all the coding.
The third step was to invent and implement a service resolver. This is also complete.
The fourth step is to test all this and collect feedback. This is what we currently do.
The next steps depend on the test-results and the feedback.
Those steps will of course contain
- documentation of the service
- creation of the necessary access tools
-
a reasonable time of parallel operation of SVN and HTTPThe goal is to completely remove SVN and only use the HTTP repository. I'd like to reach that goal better sooner than later as we currently have two single-points of failure for the SVN-service.
The first is the
http://scenery.flightgear.org/svn-service web service (runs on a private web server) and the second is the one-and-only svn-server available for public access, also hosted on a private server. If either on of those fails, the SVN-terrasync is dead. I have learned the hard way that not having a fallback is a bad idea.
I hope this clarifies the process a bit.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 2:35 pm
by SHM
Haha they won't stand a chance against Terragit.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 2:39 pm
by IAHM-COL
This is getting quite interesting.
First off, naturally http protocol does not have any indication on pre-existing data. The Hypertext transfer protocol was not developed with that in mind. I can see how you could use some sha1sum or the sort to decide if redownload changed packages, using some scripting mechanism, so that is an interesting idea. Again -- some kind of re-invention of the wheel when you compare to similar capabilities offered by either Git/subversion or alikes.
Keep in mind that once a sha1sum, or alike, determines a package has changed, with http you are due for a full download of the package itself, as opposed to a download of the diffs. Big difference, with a major impact on bandwith required to maintain
In any case, I also wonder how this works at a larger scale -- as in downloading the whole planet beforehand: which is certainly not in the mind of Torsten, whom I'd argue, likes the case scenery of not distributing the whole package completely.
Maintaining SVN along with http seems to me the correct approach. In fact terrafs does use http from libcurl already and works great for its purpose. In that sense, the only reasonable time to maintain a parallel operation, I think, is "indefinite".
Having a fallback is an awesome idea. That is reason #211 of having a git system is preferable ove a monolithic and unique svn repository. A monolithic repository of 100+GB as you have is some sort of a unconfortable joke -- which I am sure Sourceforge pointed you at when you got the space to load that thing.
Again, just to clarify, I am not suggesting that having http mirrors is a bad idea for snapshots. I am suggesting it is a rather primary and innocent idea for a development sandbox. And certainly not a great idea to massively distribute changing material (as terrasync): due to the inherent disabilties of tracking what's available and what's changing. In that sense, maintaining your svn is the second best alternative you have. In my opinion, the best one is migrating the development all-together to terraGIT and mirroring terraGIT content via http mirrors as you plan --an use those http mirrors to feed running FG.
Re: The Future of Terrasync ...
Posted: Wed May 04, 2016 4:31 pm
by OPFOR77
IAHM-COL wrote:First off, naturally http protocol does not have any indication on pre-existing data. The Hypertext transfer protocol was not developed with that in mind. I can see how you could use some sha1sum or the sort to decide if redownload changed packages, using some scripting mechanism, so that is an interesting idea. Again -- some kind of re-invention of the wheel when you compare to similar capabilities offered by either Git/subversion or alikes.
They were going to implement rsync (or something like it) to see if the data has been updated.