Intland's free requirements, development and test management hosting.
This server hosts 100.000+ users on the cloud!
Add subrepository support#11871/v12
more
Tags:  not added yet

Add subrepository support[BUG-11871]

Tracker: Bugs Priority: NormalNormal Status: Reopened
Submitted by: asafm May 05, 2010 10:22 Modified by: johnpeb Sep 13, 2011 12:30 Assigned to: ilya_ivanov, johnpeb, npiguet Feb 25, 2011 01:02
Category: Core Severity: Major Resolution: In Development
Release: -- Detected: --
Relations
Loading...
Description
Hi,

We have 4 repositories in our product (foo): foo-parent foo foo-backend foo-commons

foo-parent repository includes the three repositories as sub repositories (using the .hgsubstate file).

Also each repository is a maven project, in that hierarchy. Each sub-repository is a module inside foo-parent.

Every now and then, on save, the following error message pops:

abort: path 'foo-backend/src/main/java/com/acme/foo/app/customers/services/CustomerService.java' is inside repo 'foo-backend'. Command line: /home/asaf/work/food/foo-build-2.2.0/foo-parent/hg -y status -marduc -- /home/asaf/work/acme/foo-build-2.2.0/foo-parent/foo-backend/src/main/java/com/acme/foo/app/customers/services/CustomerService.java

The exception stack trace is:

abort: path ...
at com.vectrace.MercurialEclipse.commands.AbstractShellCommand.executeToStream(AbstractShellCommand.java:346)
at com.vectrace.MercurialEclipse.commands.HgCommand.executeToStream(HgCommand.java:156)
at com.vectrace.MercurialEclipse.commands.AbstractShellCommand.executeToBytes(AbstractShellCommand.java:287)
at com.vectrace.MercurialEclipse.commands.AbstractShellCommand.executeToBytes(AbstractShellCommand.java:276)
at com.vectrace.MercurialEclipse.commands.AbstractShellCommand.executeToBytes(AbstractShellCommand.java:272)
at com.vectrace.MercurialEclipse.commands.AbstractShellCommand.executeToString(AbstractShellCommand.java:485)
at com.vectrace.MercurialEclipse.commands.HgStatusClient.getStatusWithoutIgnored(HgStatusClient.java:145)
at com.vectrace.MercurialEclipse.team.cache.MercurialStatusCache.refreshStatus(MercurialStatusCache.java:1131)
at com.vectrace.MercurialEclipse.team.cache.MercurialStatusCache.access$0(MercurialStatusCache.java:1087)
at com.vectrace.MercurialEclipse.team.cache.MercurialStatusCache$ProjectUpdateJob.updateProject(MercurialStatusCache.java:131)
at com.vectrace.MercurialEclipse.team.cache.MercurialStatusCache$ProjectUpdateJob.run(MercurialStatusCache.java:113)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

Session Data:

eclipse.buildId=
java.version=1.6.0_12
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments:  -product org.eclipse.epp.package.jee.product
Command-line arguments:  -os linux -ws gtk -arch x86_64 -product org.eclipse.epp.package.jee.product

The IDE becomes unworkable with this bug.

Netbeans handles this configuration quite well (but I don't like it:)

Thanks!

Asaf

Details
Comments & Attachments (27)
Associations (5)
Children (0)
References (0)
SCM Commits (0)
All (0)

Filter
Submitted Comment
hgeclipse
Jan 04, 2011 05:24
Just fixed the refresh/decorator issue, without to verifying if subrepo stuff is working or not.
hgeclipse
Jan 04, 2011 04:02
There is a serious issue with this "enable subrepos support" option - if enabled on startup, decorator doesn't show any state for any file and "team -> refresh" does not work as expected, at least on MercurialEclipse project itself.

The hg console shows that only

hg -y resolve -l

is invoked if the "subrepos"  is on.

If it is "off", right commands are executed and status is properly shown:

hg -y status -marduc
hg -y id -ib --debug
hg -y resolve -l

So it is still "experimental".

Regards,

Andrei 

johnpeb
Dec 14, 2010 14:21
I just uploaded a snapshot based on the latest default (16d0dc9ca19e) here: http://mercurialeclipse.eclipselabs.org.codespot.com/hg.wiki/update_site/snapshots

subrepos is merged to default. Still to do is item 3 below - can we remove the subrepos config option. Also we need to test.

lorantp
Dec 14, 2010 08:19
Is there any progress on this? Is there a way to try the functionality out, maybe an update site?

Also, will the changes in the subrepos branch fix only the scenario when every nested project has its own sub-repo, or will there also be support for handling nested modules in a single repo?

npiguet
Oct 21, 2010 11:24
Just a note: the MercurialRootCache problems are not specific to the subrepos branch IIRC, that code just used to be inside HgRootClient. Therefore the default branch probably suffers from the same problem (something to remember if we don't merge the subrepos branch with the default branch in the end).
npiguet
Oct 21, 2010 09:59
I've used it on some test projects. Those projects are never that big though. I haven't tested much of the functionality that happens in fancy extensions (transplant, queues, etc...). I have however checked the whole code (including those extensions) to verify that when the code calls resource.getProject(), it never uses the project to figure out which HgRoot the resource belongs (this was a common anti-pattern used pretty much everywhere in the code) to but uses the resource itself.

I think the point of the protection switch was because my original implementation was too slow. However since my refactoring of the HgRoot cache and some changes in the MercurialStatusCache those performance problems should have disappeared. I was wainting on Alexei to tell me how he tests the peformance of the plugin to be able to test the peformance more thoroughly. If the performance hit proves to be small enough, I think the switch can be removed altogether.

johnpeb
Oct 16, 2010 22:11
Looks good. I think this should be the next big feature (either in this or the next release).

I think there are some issues:

  1. There are a large number of HgRoot instances created (one per file queried). We should reuse HgRoot instances
  2. Can we cache roots on IResources (org.eclipse.core.resources.IResource.setSessionProperty(QualifiedName, Object)). Currently MercurialRootCache.byFile isn't cleaned up.
  3. Do these features need to be protected by the subrepo config option or can that option be removed?
johnpeb
Oct 14, 2010 10:38
Hi Nicolas, I will try to look at this this week. I was wondering have you been using personaly a build from this branch? / How much testing has this branch had?
hgeclipse
Oct 11, 2010 15:26
Hi Ilya, John,

can you please review the Nicolas code? I have no time to work on it.

It would be nice to have it in 1.7, but it would be too bad to break something just before the 1.7 release.

Regards, Andrei

npiguet
Oct 11, 2010 08:48
Subrepository implementation should work relatively well with all commands now. Implementation has been finalized in the subrepos branch. As of changeset 2303:cf950ca55039 it should be fast an stable enough to include merge back into the default branch.
npiguet
Jun 18, 2010 10:04
I have a few ideas on how to fix performance of the subrepo stuff. Could you please tell me precisely how you test performance? It seems that you use Django as a test case (seems close enough to what we have in my company in terms of size), but how do you measure those "20 seconds" that you talk about in the commit comments?
hgeclipse
Jun 06, 2010 15:21
Update site:
http://hge.javaforge.com/mercurialeclipse-snapshots 
asafm
Jun 06, 2010 09:33
Is there an Eclipse Update Site containing the fix you've made? I want to give it a test drive.
hgeclipse
May 23, 2010 13:23
So I've tried this changes, and I have somewhat mixed feeling.

It works only if the parent project is not opened. If it is opened, MercurialEclipse starts to behave strange - sometimes refresh doesn't work, some actions (commit/revert etc) are not working etc, because plugin all the time mixes the hg roots.

So as a result I've added a new "performance" option to enable the subrepositories support and madeit disabled per default - because it is not mature/stable/fast enough.

@Nicolas: 

It would be nice if you could start a separated branch dedicated to this feature.

The right solution should maintain the map/state of existing hg roots/subrepos. This map should be updated on project open/close events and on Eclipse startup. Neither decorator nor status cache should permanently search for subrepositories in folders, as this decreases performance on huge repos - and huge repos ARE mostly managed with subrepos. All the code which is now calculating the hg roots for resources (MercurialTeamProvider etc) should take the subrepos into the account. Probably the best way would be to extend HgRoot class and put there references to the subrepos/parent repo and ask this in the related low level hg code (commit/revert etc).

Regards,

Andrei 

npiguet
May 18, 2010 04:03
I've merged the whole thing with the latest changeset on the bitbucket repository. It should be fine to pull it into the javaforge repos.
npiguet
May 15, 2010 08:18
I've just pushed a correction. It should be faster this way (and the code is actually simpler).
hgeclipse
May 15, 2010 06:49
Hi Nicolas,

regarding your changes: the lines below in ResourceDecorator:

if (resource.getType() == IResource.PROJECT || resource.getType() == IResource.FOLDER && AbstractClient.isHgRoot(resource) != null) {
				suffix = getSuffixForContainer((IContainer)resource);
}

 

slows down the decorator a lot because AbstractClient.isHgRoot(resource) tries to resolve canonical folder path TWO times EACH time a folder is decorated (it uses HgRootClient.hasHgRootDirect() -> getHgRootFromCache() and hasHgRootDirect() itself).

Both times are two times too much. I would NOT use getCanonicalPath(file) for decorator at all.

Can you please take a look at it? 

Regards,

Andrei 

hgeclipse
May 15, 2010 05:48
 IResource from an IPath, and to make sure to always get the same IResource instance for equal IPaths

Please take a look at ResourceUtils.getFileHandle(IPath) and ResourceUtils.convert(File). The general answer is - yes and no.

IPath is just a *path* , and can be converted (properly) to the IResource if it exists in the workspace AND Eclipse KNOWS that (workspace is up-to-date with file system state). 

Additionally, there can be more then one IResource instance matching same IPath - in case of "nested" projects.

Hope this helps. 

npiguet
May 14, 2010 20:43
I consider the whole thing to work satisfactorily from commit 2072:1d99f7e77c66
npiguet
May 14, 2010 18:08
As of changeset c95aaa022202, the crash is fixed and the status is correctly updated and refreshed in subrepos.

I'd like to go a little bit further, but for that, I need to know if its possible to get an IResource from an IPath, and to make sure to always get the same IResource instance for equal IPaths (this is required to store the merge changeset id on any folder just as it is done for projects.)

Only the last 20 comment/attachment loaded. Show All (27)