AN INDEX FOR COUNTING FIXED POINTS

OF AUTOMORPHISMS OF FREE GROUPS

Damien Gaboriau, Andre Jaeger, Gilbert Levitt, Martin Lustig

Abstract. Let α be an automorphism of a free group F of rank n. The Scott conjecture, proved by Bestvina-Handel, states that the fixed subgroup Fix α = {g ∈ F | α(g) = g} has rank at most n. Using R-trees, we show the stronger inequality rk Fix α + \frac{1}{2} a(α) ≤ n, where a(α) is the number of Fix α-orbits of attracting fixed points for the action of α on the boundary of F.

Introduction

Let α be an automorphism of F = F_{n}, the free group of rank n. The Scott conjecture, proved by Bestvina-Handel [BH], states that the fixed subgroup Fix α = {g ∈ F | α(g) = g} has rank at most n.

We shall improve this result by showing:

Theorem 1. If α is any automorphism of F_{n}, then rk Fix α + \frac{1}{2}a(α) ≤ n.

Here a(α) is the number of equivalence classes of attracting fixed points for the action of α on the boundary of F (defined below). This answers positively a conjecture of Cooper’s ([Co, p. 455]).

If Fix α is trivial, our result specializes to:

Corollary. An automorphism α of F_{n} with Fix α = {1} fixes at most 4n ends of F_{n}.

To define a(α) in general, we consider the boundary δF of F (see Part I), the Cantor set of ends of F if n ≥ 2. If we choose a free basis g_{1}, …, g_{n}, it may be viewed as the set of all infinite reduced words X = x_{1} ⋯ x_{i} ⋯ in the letters g_{j}^{±1}. The action of α on F extends to a continuous action of α on δF. The boundary of the subgroup Fix α naturally embeds in δF, and α acts on δ(Fix α) as the identity.

We consider fixed points of α in δF. It turns out (Proposition I.1) that such a fixed point X either belongs to δ(Fix α), or is attracting, or is repelling (i.e. attracting for α^{−1}). Here attracting may be understood in the topological sense (lim_{p→+∞} α^{p}(X′) = X for X′ close to X in F ∪ δF), or in the algebraic sense of [CL 1, 1.4]. As in [CL 1], we say that two fixed points X_{1}, X_{2} ∈ δF are equivalent if there exists g ∈ Fix α such that X_{2} = gX_{1}. Note that any point equivalent to an attracting fixed point of α is itself an attracting fixed point of α.

Typeset by AMS-TEX

We let A(α) be the set of equivalence classes of attracting fixed points of α, and we denote a(α) the cardinality of A(α). Finiteness of a(α) follows from [Co] (or [CL 1]).

Theorem 1 may be illustrated by the following example from [CL 1]. Let α : F_{2} → F_{2} be given by α(a) = aba, α(b) = ba. The fixed subgroup has rank 1, it is generated by aba^{−1}b^{−1}. One obtains two inequivalent fixed words X_{1} = ababaaba ⋯ and X_{2} = a^{−1}b^{−1}a^{−1}a^{−1}b^{−1}a^{−1}b^{−1}a^{−1} ⋯ by taking the limit as p goes to +∞ of α^{p}(a) and α^{p}(a^{−1}) respectively. Note that X_{3} = baabaaba ⋯ = lim_{p→∞} α^{p}(b) is equivalent to X_{1}. The automorphism α is induced by a pseudo-Anosov homeomorphism φ of a punctured torus M. The fixed subgroup corresponds to the boundary of M, while the equivalence classes of attracting fixed points correspond to the separatrices of the unstable foliation of φ.

This paper elaborates on the note [GLL], which contains a proof of the Scott conjecture based on R-trees. As in [GLL], the two main ingredients in this paper are the existence of a certain α-invariant R-tree, and an inequality by Gaboriau-Levitt [GL] about stabilizers of branch points in R-trees.

Given an automorphism α of F, there is an action of F on an R-tree T, which is α-invariant in the following sense: its length function ℓ satisfies ℓ ∘ α = λℓ for some number λ ≥ 1 (this action represents a fixed point for the action of α on the boundary of Culler-Vogtmann’s outer space). Equivalently, there exists a homothety H : T → T with stretching factor λ ≥ 1 (i.e. d(Hx, Hy) = λd(x, y) for x, y ∈ T), such that α(w)H = Hw for all w ∈ F (we identify an element of F and the associated isometry of T).

The existence of such an invariant tree, with a very small action of F, is now well-known (Bestvina-Handel, Skora, Lustig [Lu], Paulin [Pa 3]). In Part II we shall construct T and derive additional properties. In particular we prove the fact (due to Lustig [Lu]) that T may be assumed to have trivial arc stabilizers. We also prove that H has a fixed point Q ∈ T whenever rk Fix α + \frac{1}{2}a(α) > 0.

In Part IV we analyze the invariant tree T, relating properties of α to geometric properties of T. This may be viewed as the heart of the paper. We distinguish several cases. Here we mention only the most interesting one, when λ > 1. In this case we use the fact (proved in Part III using an estimate by Bestvina-Feighn-Handel for maps between metric graphs) that every R-tree with a very small action of F has the following Bounded BackTracking property (BBT): Given Q ∈ T, there exists C > 0 such that, if v, w are reduced words in F with no cancellation in the product vw, then d(vQ, [Q, vwQ]) ≤ C, where [Q, vwQ] denotes the segment between Q and vwQ.

Consider an attracting fixed point of α, represented by a fixed infinite word X = x_{1} ⋯ x_{i} · · ·. We write X_{p} = x_{1} ⋯ x_{p}. Applying (BBT) to the fixed point Q of H we show that either X ∈ δ(Stab Q) ⊂ δF (where Stab Q ⊂ F is the stabilizer of Q), or as p goes to +∞ the sequence X_{p}Q goes to infinity in T, staying at a bounded distance from an H-invariant infinite ray ρ starting at Q.

This leads to a injection from the set of equivalence classes of attracting fixed points of α not contained in δ(Stab Q), to the set of orbits of the action of Stab Q on π_{0}(T \ {Q}).

We now use Theorem III.2 of [GL], which in the special case when arc stabilizers are trivial may be stated as follows:

Theorem 2 [GL]. Let T be an R-tree with a minimal F_{n}-action whose arc stabilizers are all trivial. Given Q_{1}, …, Q_{q} ∈ T belonging to distinct orbits, we have

∑_{ℓ=1}^{q}(rk Stab Q_{ℓ} + \frac{1}{2}v(Q_{ℓ}) − 1) ≤ n − 1,

where v(Q_{ℓ}) ≥ 1 is the number of orbits of the action of Stab Q_{ℓ} on π_{0}(T \ {Q_{ℓ}}). In particular, rk Stab Q ≤ n − 1 and v(Q) ≤ 2n for every Q ∈ T.

Using Theorem 2, we prove Theorem 1 by induction on n in Part V. As in [GLL], we need to consider several automorphisms simultaneously to make the induction work.

Two automorphisms α, β of F represent the same outer automorphism Φ ∈ Out (F) if there exists m ∈ F such that β = i_{m} ∘ α, with i_{m}(g) = mgm^{−1}. As in [GLL], we say that α and β are similar if m can be written m = cα(c^{−1}) for some c ∈ F, or equivalently if β = i_{c} ∘ α ∘ (i_{c})^{−1}. Similar automorphisms represent the same automorphism, up to a change of basis in F. In particular the rank of Fix α, and the number a(α), are similarity invariants.

What we actually prove by induction on n in Part V is the following statement:

Theorem 1’. Let α_{0}, …, α_{k} be automorphisms of F_{n} representing the same outer automorphism and belonging to distinct similarity classes. Then

∑_{i=0}^{k}(rk Fix α_{i} + \frac{1}{2}a(α_{i}) − 1) ≤ n − 1.

Equivalently: ∑_{α∈S(Φ)} max(0, rk Fix α + \frac{1}{2}a(α) − 1) ≤ n − 1, where S(Φ) is the set of similarity classes of automorphisms representing a given outer automorphism Φ ∈ Out (F).

(This theorem is only superficially stronger than Theorem 1. It follows from Theorem 1 by applying it to the automorphism of F_{n} ∗ F_{k} equal to α_{0} on F_{n} and sending the i-th generator t_{i} of F_{k} to t_{i}u_{i} where α_{i} = i_{u_{i}} ∘ α_{0}.)

As an example, let M be a compact orientable surface of genus 2 with one boundary component. Let φ be a pseudo-Anosov homeomorphism of M whose unstable foliation has two singularities, a 7-prong saddle x in the interior of M and a tripod y on the boundary (see [MaSm, Theorem 2] for a proof of existence). The homeomorphism φ induces two non-similar automorphisms α_{x} and α_{y} of F_{4} ≃ π_{1}M, depending on whether the basepoint is set at x or y (note that x and y are fixed by φ but belong to different Nielsen classes). One has Fix α_{y} ≃ Z and a(α_{y}) = 1 (corresponding to the infinite separatrix of y). On the other hand Fix α_{x} is trivial and a(α_{x}) = 7. In this example, equality holds in Theorem 1’.

In Part VI, written rather informally, we discuss and interpret the index

ind (α) := rk Fix α + \frac{1}{2}a(α) − 1

of an automorphism of F, as well as the index

ind (Φ) := ∑_{α∈S(Φ)} max(0, rk Fix α + \frac{1}{2}a(α) − 1)

of Φ ∈ Out (F) that appears in Theorem 1’. It turns out that ind (Φ) is strongly related to important structural properties of Φ. In particular we will consider automorphisms of maximal index n − 1.

I. The action of α on δF

Let F be a free group of rank n and \overline{F} its compactification (end completion as in [Co], or compactification as a hyperbolic group, see e.g. [Sho 1, ch. 4]). The boundary δF = \overline{F} \ F is a compact space, homeomorphic to a Cantor set if n ≥ 2. The natural actions of F and Aut (F) on F extend continuously to \overline{F}. Any finitely generated subgroup F′ ⊂ F is quasiconvex [Sho 2], hence inclusion induces a natural embedding δF′ → δF (see [CDP, p. 115]).

Once we fix a free generating system g_{1}, …, g_{n} for F, we view F as the set of reduced words in the letters g_{j}^{±1}, and δF as the set of infinite reduced words X = x_{1} ⋯ x_{i} · · ·. We denote X_{i} = x_{1} ⋯ x_{i}.

Given two reduced words X, X′, finite or infinite, we let X∧X′ be their common initial segment (the empty word if x_{1} ≠ x′_{1}) and we denote c_{X′}(X) the length of X ∧ X′. A sequence of reduced words X_{p} ∈ \overline{F} converges to X ∈ δF if and only if lim_{p→+∞} c_{X}(X_{p}) = +∞.

Now let α be an automorphism of F. Recall ([Co], see Part III) that there is a cancellation bound for α, that is a number B such that

|α(vw)| ≥ |α(v)| + |α(w)| − 2B

whenever v, w, vw are finite reduced words with |vw| = |v| + |w| (| | denotes word length).

Let X = x_{1} ⋯ x_{i} ⋯ be fixed by α. We write α(X_{i}) = X_{k(i)}Z_{i} with k(i) = c_{X}(α(X_{i})). Since X is fixed by α, the sequence k(i) goes to +∞ as i increases. Bounded cancellation implies |Z_{i}| ≤ B. Also note that |k(i + 1) − k(i)| is bounded by a constant depending only on α (namely max |α(g_{i})|).

As in [CL 1], we say that X is an attracting fixed word of α if

lim_{i→+∞}(k(i) − i) = +∞.

Note that there exists i_{0} such that for all i ≥ i_{0} one has k(i) ≥ i + B + 1 and hence

c_{X}(X′) ≥ i_{0} ⟹ c_{X}(α(X′)) > c_{X}(X′).  (∗)

We say that X is an attracting fixed point of α if there exists a neighborhood U of X in \overline{F} such that

X′ ∈ U ⟹ lim_{p→+∞} α^{p}(X′) = X.

The following proposition shows that the two notions of attraction are the same. We say that X is repelling for α if it is attracting for α^{−1}.

Proposition I.1. Let X ∈ δF be an infinite fixed word of α. The following are equivalent:

(1) X is an attracting or repelling fixed word.

(2) X is an attracting or repelling fixed point for the action of α on \overline{F}.

(3) X ∉ δ(Fix α). (of course the subcases attracting or repelling in (1) and (2) coincide).

Proof. (1) ⟹ (2): Suppose X is an attracting fixed word. Choose i_{0} as in (∗). If c_{X}(X′) ≥ i_{0}, we get c_{X}(α(X′)) > c_{X}(X′), hence lim_{p→+∞} c_{X}(α^{p}(X′)) = +∞ and lim_{p→+∞} α^{p}(X′) = X.

(2) ⟹ (3): this is clear since α acts as the identity on \overline{Fix α} and no point X ∈ δ(Fix α) can be isolated in \overline{Fix α}.

To prove (3) ⟹ (1), let X be any infinite fixed word. As in [Co], we consider the words w_{i} = X_{i}^{−1}α(X_{i}). We note that

w_{i} = w_{p} ⟹ X_{p}X_{i}^{−1} ∈ Fix α.

If the sequence of words w_{i} takes the same value infinitely often we get X ∈ \overline{Fix α} since for fixed i we have X = lim_{p→+∞} X_{p}X_{i}^{−1}.

Otherwise |w_{i}| goes to infinity. Recall that α(X_{i}) = X_{k(i)}Z_{i} with |Z_{i}| ≤ B and |k(i + 1) − k(i)| bounded. Since |w_{i}| is comparable to |k(i) − i| (the difference is bounded by B) we see that k(i) − i goes to either +∞ or −∞ as i increases. If the limit is +∞, then X is an attracting fixed word. If it is −∞, we repeat the argument for α^{−1}. Define \overline{k}(i) and \overline{Z}_{i} analogously to k(i) and Z_{i}. Writing X_{i} = α^{−1}(X_{k(i)})α^{−1}(Z_{i}) we see that \overline{k}(k(i)) − i stays bounded. Then \overline{k}(k(i)) − k(i) = (\overline{k}(k(i)) − i) − (k(i) − i) goes to +∞, and so does \overline{k}(i) − i (recall that k(i) goes to +∞). This means that X is a repelling fixed word.

Note that the corollary stated in the introduction is an immediate consequence of Theorem 1: if Fix α = {1}, then there are at most 2n attracting fixed points and 2n repelling ones.

II. The invariant tree

A. Statement of the result.

For the convenience of the reader we state some basic definitions about R-trees. For more detailed information see e.g. [MoSh], [CM] or the survey articles [Sha 1], [Sha 2].

An R-tree is a non-empty metric space in which any two distinct points x, y are joined by a unique arc [x, y], and in which every such arc is isometric to a closed interval of R (recall that a (nondegenerate) arc is a space homeomorphic to [0, 1]).

Alternatively, an R-tree is a path-connected metric space (X, d) which satisfies the 0-hyperbolicity condition

d(x, z) + d(y, w) ≤ max{d(x, y) + d(z, w), d(x, w) + d(z, y)},

see [GH].

We consider R-trees equipped with a left action of a group G by isometries. The stabilizer Stab (x) of a point x ∈ T is the subgroup of G consisting of elements fixing x. Similarly, the stabilizer of an arc [x, y] is the subgroup consisting of elements fixing [x, y] pointwise.

The G-action on T (or sometimes just T) is called

– trivial ⟺ ∃ x ∈ T with Stab (x) = G,

– simplicial ⟺ T arises from a simplicial tree by setting each edge length equal to some positive value (always 1 in this paper),

– minimal ⟺ there is no proper G-invariant subtree,

– free ⟺ the stabilizer of every point is trivial,

– with trivial arc stabilizers ⟺ the stabilizer of every arc [x, y] is trivial,

– small ⟺ no arc stabilizer contains a free group of rank 2.

Let α be an automorphism of F. Recall that the index of α is the quantity

ind (α) = rk Fix α + \frac{1}{2}a(α) − 1

that appears in Theorem 1’. It is an integer, or a half-integer, with ind (α) ≥ −1. Our interest will be in automorphisms with positive index.

We now state the main result of this section which is proved in subsections B to E. A homothety of a metric space (X, d) is a map H : X → X satisfying d(Hx, Hy) = λd(x, y) for some fixed λ > 0 called the stretching factor.

Theorem II.1. For every automorphism α of F there exists an R-tree T such that:

(1) F acts on T non-trivially, minimally, with trivial arc stabilizers.

(2) There exist λ ≥ 1 and a homothety H: T → T with stretching factor λ

such that

α(w)H = Hw: T → T

for all w ∈ F. If λ = 1, then T is simplicial.

(3) If ind (α) > 0, then H has at least one fixed point Q ∈ T. More generally, if β = i_{m} ∘ α satisfies ind (β) > 0, then H_{β} = mH has a fixed point (recall that i_{m}(g) = mgm^{−1}).

Remark II.2. It is easy to see that, given α and T, the equation α(w)H = Hw uniquely determines H (and λ). The second part of Assertion 3 will be used when we study several automorphisms simultaneously (see Part V). Note that β(w)H_{β} = H_{β}w for all w ∈ F, since

β(w)H_{β} = mα(w)m^{−1}mH = mα(w)H = mHw = H_{β}w.

Thus we may use the same tree T to study all automorphisms representing a given outer automorphism Φ. Indeed, Assertion 2 is equivalent to the equation ℓ ∘ Φ = λℓ, where ℓ is the length function of the action of F on T.

B. A criterion for a fixed point.

Let τ be a finite connected graph, with π_{1}τ free of rank n ≥ 2. We fix a universal covering τ̃ → τ, and an isomorphism from the group of covering transformations to F (this is a way of identifying π_{1}τ with F without having to choose a basepoint; of course this identification is only defined up to a conjugacy in F).

Let f : τ → τ be a homotopy equivalence. It induces a well-defined outer automorphism Φ of F, and the formula

α(w)f̃ = f̃w  (∗∗)

defines a 1-to-1 correspondence between the set of lifts f̃ : τ̃ → τ̃ and the set of automorphisms α of F representing Φ. We fix an automorphism α and the corresponding f̃.

The following result generalizes Lemma 2.1 of [BH].

Proposition II.3. Let f̃: τ̃ → τ̃ be associated with the automorphism α of F by (∗∗). If ind (α) > 0, then f̃ has a fixed

point.

Proof. As in [BH, p. 19], we shall use the following geometric fixpoint criterion for f̃: Let x, y ∈ τ̃ be distinct points with the property that f̃(x) is distinct from x and is not contained in the same connected component of τ̃− {x} as y, and conversely. Then f̃ has a fixed point on [x, y].

Make τ into a metric graph by declaring that every edge has length 1, and equip τ̃ with the lifted metric. For any point P ∈ τ̃, the map j : w ↦ wP gives a quasiisometric embedding F → τ̃. This induces a homeomorphism between δF and the space δτ̃ of ends of τ̃, which is independent of the choice of P. Furthermore, the distance in τ̃ between f̃(j(w)) = α(w)f̃(P) and j(α(w)) = α(w)P is bounded by the distance between P and f̃(P), independently of w ∈ F. It follows that the extension of f̃ to δτ̃ agrees with the extension of α to δF, and an attracting fixed point of α in δF defines an attracting fixed point for f̃ in δτ̃.

In the situation of Proposition II.3, first assume that α has at least two distinct (possibly equivalent) attracting fixed points X_{1}, X_{2} in δF. Then any two points x, y ∈ τ̃ which are sufficiently close to the corresponding fixed points on δτ̃ satisfy the hypothesis of the above criterion and f̃ has a fixed point.

Such points X_{1}, X_{2} exist if a(α) ≥ 2. They also exist if there exist both an attracting fixed point X_{1} in δF and a nontrivial α-fixed u ∈ F, as we can then take X_{2} = uX_{1}. Since we assume ind (α) > 0, the only remaining case is when rk (Fix α) ≥ 2. But then Lemma 2.1 of [BH] applies.

C. Partial train track maps.

There have been various independent attempts (see [BH], [Lo], [Lu]) to carry Thurston’s concept of train tracks for surface homeomorphisms over to automorphisms of free groups. Our notion of partial train track maps in Definition II.4 is close to but weaker than the “relative train track maps” in [BH, ch.5]. Below we crucially use the existence of such a relative train track map for every outer automorphism of F, as shown in [BH, Theorem 5.12]. Alternatively one can use §5 of [LuOe].

Given f : τ → τ as above, we say that a locally injective path c : [0, 1] → τ from a point p to a point q is f-backtracking if f(p) = f(q) and the loop f ∘ c is null-homotopic in τ. Equivalently, any lift of c to the universal covering τ̃ is a segment [p̃, q̃] with f̃(p̃) = f̃(q̃).

Definition II.4. A continuous map f: τ → τ is called a partial train track map relative to τ′ ⊂ τ if the following conditions are satisfied:

(1) τ is a finite connected graph with no vertices of valence 1.

(2) f is a homotopy equivalence.

(3) f preserves the set τ^{0} of vertices of τ : f(τ^{0}) ⊂ τ^{0}.

(4) τ′ is a (not necessarily connected) subgraph of τ which satisfies: (a) τ′ ≠ τ

(b) f(τ′) ⊂ τ′

(c) τ′ ∪ τ^{0} is maximal with respect to (a) and (b).

(5) For any k ≥ 1, all f^{k}-backtracking paths which are contained in τ −(τ′∪τ^{0}) are mapped by f^{k} to τ′.

Lemma II.5. Let f : τ → τ be a relative train track map in the sense of [BH], with f-invariant maximal filtration τ_{0} ⊂ … ⊂ τ_{m}. If τ has no vertices of valence 1, then f is a partial train track map relative to τ′ = τ_{m−1}.

Proof. Condition (1) of Definition II.4 holds by assumption. Conditions (2) and (3) hold, as f is a topological representative, see [BH, pp. 4-5]. Condition (4) follows from our definition of τ′. If the m-th stratum of τ is exponentially growing, then Condition (5) follows from Lemma 5.8 of [BH]. If this stratum is not exponentially growing, then the transition matrix M(f) is a permutation matrix and hence, for every k ≥ 1, the f^{k}-image of any edge e ∈ τ − τ′ contains only one edge of τ − τ′. This implies Condition (5).

Proposition II.6. Given an outer automorphism Φ of F, there exists a partial train track map f: τ → τ which induces Φ.

Proof. By Theorem 5.12 of [BH], there exists a relative train track map f: τ → τ for Φ. By Lemma 5.2 of [BH] we can modify

f and τ so that τ doesn’t contain any vertex of valence one. Now apply Lemma II.5.

Now let f : τ → τ be a partial train track map. Let e_{1}, …, e_{p} be the (unoriented) edges of τ −τ′ (or more precisely of the closure of τ −τ′). Conditions (3) and (5) of Definition II.4 imply that for any k ≥ 1 the f^{k}-image of any edge e_{i} ⊂ τ −τ′ crosses properly over the edges of τ − τ′. Hence f determines a p × p transition matrix M(f) with non-negative integer entries, where the i-th row records the number of times the f-image of e_{i} crosses the edges e_{1}, …, e_{p}, disregarding the orientation.

¿From condition (4c) it follows that M(f) is either zero, or irreducible with Perron-Frobenius eigenvalue λ ≥ 1 (for definitions and the Perron-Frobenius Theorem, see [Ga] or [DV]). As Conditions (1) and (2) exclude the possibility of M(f) being zero, we have λ ≥ 1.

Let v = (v_{i}) be a strictly positive eigenvector v associated to the eigenvalue λ (it exists and is unique up to scaling). We define the PF-length of an edge e_{i} of τ − τ′ as L(e_{i}) = v_{i} (and we set L(e) = 0 for all edges e ⊂ τ′). Note that the image by f of any edge e is a path of PF-length λL(e).

Let τ̄ be the tree obtained from the universal covering τ̃ by collapsing every connected component of the preimage of τ′ to a point. The action of F on τ̃ induces an action on τ̄ which is not necessarily free but which has trivial arc stabilizers. Make τ̄ into a metric tree by lifting the PF-length L to the edges of τ̄. We get an F -invariant distance function d: if x, y ∈ τ̄, then d(x, y) is the PF-length of the segment [x, y].

Remark. If the automorphism α is irreducible (in the sense of [BH]), then there exists an absolute train track representative for α, i.e. a train track representative f: τ → τ with only one stratum. In this important special case f is a partial train track map relative to τ′ = ∅, and τ̄ = τ̃.

Given α ∈ Aut (F), let f be a partial train track map inducing the outer automorphism determined by α. Let f̃: τ̃ → τ̃ be the lift of f associated to α by (∗∗) (see the beginning of II.B).

Since f(τ′) ⊂ τ′, the map f̃ induces f̄: τ̄ → τ̄. By Condition (5) of Definition II.4, the image by f̄ of an edge e is a segment of length λL(e). We may then redefine f̄ on the interior of e, so that f̄|e expands uniformly by λ. We do this equivariantly, thus making sure that the relation α(w)f̄ = f̄w still holds. The new map (also denoted f̄)

satisfies d(f̄(x), f̄(y)) ≤ λd(x, y), and equality holds if x, y belong to the same edge.

D. The case λ = 1.

If λ = 1, then M(f) is a permutation matrix, and we can choose the eigenvector v to have all entries v_{i} = 1. Since f permutes the edges of τ −τ′, the map f̄: τ̄ → τ̄ is a homeomorphism and therefore a global isometry (no folding occurs).

We claim that T = τ̄ satisfies the conclusions of Theorem II.1. Conditions (1) and (2) obviously hold, with H = f̄. If ind (α) > 0, the map f̃ has a fixed point q ∈ τ̃ by Proposition II.3. The image Q of q in T is a fixed point of H. To get a fixed point for H_{β} = mH, we simply apply this argument to the map mf̃, noting that it satisfies (∗∗) with respect to β.

E. The case λ > 1.

The rest of Part II will be devoted to the proof of Theorem II.1 when λ > 1. In this case we first have to replace the distance d by a pseudo-distance d_{∞} for which f̄ acts as a homothety (recall that f̄ only satisfies the inequality d(f̄(x), f̄(y)) ≤ λd(x, y)). We present this construction with a fairly high degree of generality.

A pseudo-distance on a set Z is a function θ: Z × Z → R^{+} which satisfies the axioms of a distance, except that there may be distinct points of Z which have distance 0. The function θ induces a genuine distance on the set Z(θ) obtained from Z by identifying x, y whenever θ(x, y) = 0. We will denote ψ : Z → Z(θ) the canonical quotient map.

Let Z be a set equipped with a distance d (a pseudo-distance would in fact be enough). Let λ > 0, and let h: Z → Z be any map which satisfies a Lipschitz condition d(h(x), h(y)) ≤ λd(x, y) for all x, y ∈ Z.

Let d_{∞} be the limit of the non-increasing sequence of pseudo-distances

d_{k}(x, y) = \frac{d(h^{k}(x), h^{k}(y))}{λ^{k}}.

It obviously satisfies d_{∞}(x, y) ≤ d(x, y) and d_{∞}(h(x), h(y)) = λ d_{∞}(x, y) for all x, y ∈ Z.

The map H induced by h on the associated metric space Z(d_{∞}) is thus a homothety with stretching factor λ.

Now assume that a group G acts on Z isometrically with respect to d, and that there exists a map Δ: G → G (usually an automorphism) which “commutes” with h in the sense that h(g(x)) = Δ(g)(h(x)) for all g ∈ G, x ∈ Z.

Then G preserves the pseudo-distance d_{∞} as well. In other words there is an induced isometric action of G on Z(d_{∞}), for which the natural surjection ψ : Z → Z(d_{∞}) is G-equivariant. Furthermore this action commutes with H in the sense that Δ(g)H = Hg for all g ∈ G. We shall also denote d_{∞} the induced distance on Z(d_{∞}).

In particular, suppose that Z is a path-connected metric space and d satisfies a δ-hyperbolicity inequality

d(x, z) + d(y, w) ≤ max{d(x, y) + d(z, w), d(x, w) + d(z, y)} + 2δ.

If λ > 1, then T = Z(d_{∞}) is 0-hyperbolic. It is an R-tree, equipped with an isometric action of G and a homothety H: T → T with stretching factor λ (T is path-connected because ψ : Z → T is continuous as d_{∞} ≤ d).

Remark. In general, it is quite possible that all points of Z have d_{∞}-distance 0 from each other, so that T consists of a single point only. In particular this is true if λ is strictly larger than the infimum of all numbers λ such that d(h(x), h(y)) ≤ λd(x, y).

Applying the above construction to the metric space Z = (τ̄, d) and the map h = f̄, we obtain an R-tree T = τ̄(d_{∞}) equipped with an isometric action of F and a homothety H satisfying Condition (2) of Theorem II.1. Condition (3) holds for the same reasons as in the case λ = 1 (of course any homothety with λ > 1 has a unique fixed point in the metric completion of T, but we insist that the fixed point be in T).

There remains to check Condition (1).

Lemma II.7. The action of F on T is non-trivial and minimal.

Proof. For each edge e_{i} of τ − τ′, choose a lift ē_{i} = [a, b] to τ̄. By Condition (5) of Definition II.4, all the backtracking in f^{k}(e_{i}) takes place in τ′. It follows that d_{k}(a, b) = \frac{d(h^{k}(a),h^{k}(b))}{λ^{k}} equals d(a, b) for all k, and the image of ē_{i} in T is a segment f_{i} of length L(e_{i}).

Note that the F -orbit of any x ∈ T meets some f_{i}. In particular, if T′ ⊂ T is an F -invariant subtree, the Hausdorff distance D(T′, T) = sup_{x∈T} d_{∞}(x, T′) is finite: it is bounded by the diameter of any subtree containing all of the f_{i}’s. If furthermore T′ is invariant under H, then D(T′, T) has to be 0 because λD(T′, T) = D(T′, T).

Now we can prove that the action is not trivial. Assume it is. The fixed subtree T_{0} = {x ∈ T | gx = x ∀g ∈ F } is compact (it is contained in the union of the f_{i}’s because F acts as the identity on T_{0} ∩ f_{i} and every orbit meets some f_{i}). Since it is H-invariant, it has to be a point. This forces T to be a point because D(T_{0}, T) = 0, a contradiction since f_{i} ⊂ T.

As the action is nontrivial, there is a unique minimal F -invariant subtree T_{1} (see [CM]), which is H-invariant. We want to show T_{1} = T. We know that D(T_{1}, T) = 0, in other words T − T_{1} may contain only endpoints of T (points x with T −{x} connected). It follows that the interior of each segment f_{i} is contained in T_{1}. Irreducibility of M(f) and λ > 1 imply that ē_{i} occurs in the interior of f̄^{k}(ē) for some edge e of τ̄ and some k ≥ 1. This proves that f_{i} is contained in H^{k}T_{1} = T_{1}, and therefore T_{1} = T.

Lemma II.8. The F -action on T has trivial arc stabilizers.

Proof. The action of F on T = τ̄(d_{∞}) is the limit of the sequence of actions (F, τ̄(d_{k})). Each of these actions is small because it has trivial arc stabilizers. Therefore the action on T is small, as a limit of small actions (see [CM], [Pa 1]).

To prove that T has trivial arc stabilizers, assume that c ⊂ T is a nondegenerate arc fixed by some nontrivial w ∈ F. Let p ≥ 1 be the largest integer such that w is a p-th power (in fact p = 1 because the action is very small in the sense of [CL 2]). Recall that the F -orbit of any x ∈ T meets some f_{i}. Since the length of H^{k}(c) grows arbitrary large with k, we can find, for sufficiently large k, disjoint non-degenerate subarcs c_{0}, …, c_{p} of H^{k}(c) such that c_{i} = v_{i}c_{0} for some v_{i} ∈ F (i = 1, …, p).

The element w′ = α^{k}(w) fixes H^{k}(c). Since w′, v_{1}, …, v_{p} all have different actions on c_{0}, there exists i such that v_{i} and w′ do not generate a cyclic subgroup of F. Then w′ and v_{i}^{−1}w′v_{i} generate a free subgroup of rank 2 which fixes c_{i} pointwise, contrary to the fact that the action is small.

Remark. The last argument may be extended to small actions of hyperbolic groups such that there exists a homothety with λ > 1 which commutes with an automorphism α (in the sense of Assertion (2) of Theorem II.1). This applies in particular to the actions constructed by Paulin [Pa 3].

III. Bounded backtracking

Let f : T_{1} → T_{2} be a continuous map between R-trees. As in Part II, a path c: [0, 1] → T_{1} is called an f-backtracking path if c is injective and f(c(0)) = f(c(1)). We say that the map f has bounded backtracking if the image (f ∘ c)([0, 1]) of any f-backtracking path c has diameter bounded independently of c.

Proposition III.1. Let T be an R-tree with a minimal action of F. The following properties are equivalent: (BBT1) Given Q ∈ T, there exists C > 0 such that, if v, w, vw ∈ F have word

length satisfying |vw| = |v| + |w|, then d(vQ, [Q, vwQ]) ≤ C. (BBT2) Given Q ∈ T, there exists C > 0 such that d((v_{1} ∧ v_{2})Q, [v_{1}Q, v_{2}Q]) ≤ C

if v_{1}, v_{2} are reduced (recall that ∧ denotes the common initial subword). (BBT3) Every F -equivariant map f: Γ̃ → T, where Γ̃ is a simplicial R-tree with a

free minimal F -action, has bounded backtracking. (BBT4) For every F -equivariant map f: Γ̃ → T, where Γ̃ is a simplicial R-tree

with a free minimal F -action, there exists a constant δ ≥ 0 such that for all x, y ∈ Γ̃ one has f([x, y]) ⊂ N_{δ}([f(x), f(y)]) (the δ-neighborhood of [f(x), f(y)]).

Notice that (BBT4) admits direct generalizations to actions of arbitrary groups G on R-trees, through replacing Γ̃

by a Cayley graph of G. Notice also that in this view BBT appears to be a kind of “one-sided quasi-isometry”.

We shall not prove Proposition III.1, as we only need the easy implication (BBT1) ⟹ (BBT2) (which is proved by setting v and w equal to the reduced words representing v_{1}^{−1}(v_{1} ∧ v_{2}) and (v_{1} ∧ v_{2})^{−1}v_{2} respectively). We will need, however, the fact that the tree T provided by Theorem II.1 satisfies (BBT1) when λ > 1. This may be derived directly from the construction of T in II.E, but we give a general argument.

Note that every free minimal action of F on a simplicial R-tree T satisfies (BBT3), as every map f as in (BBT3) is a quasi-isometry. Also note that Cooper’s cancellation bound ([Co], see Part I), is a special case of this. However, we need the following stronger version, due to Bestvina-Feighn-Handel (see [DV, Lemma II.2.4]).

Lemma III.2. If a minimal action of F on an R-tree T is free and simplicial, then it satisfies (BBT1) with C = ∑_{i=1}^{n} d(Q, g_{i}Q).

Proof. Let Z be the quotient graph of T by the action of F. Let Y be a wedge of n circles, the i-th circle having length d(Q, g_{i}Q). Let γ : Y → Z be the natural map sending the i-th circle to the projection of the segment [Q, g_{i}Q]. Then apply Lemma II.2.4 of [DV].

The action given by Theorem II.1 has trivial arc stabilizers, hence it is very small in the sense of Cohen-Lustig [CL 2].

Corollary III.3. Every very small action of F on an R-tree T has BBT.

We do not know whether this can be extended to more general actions, such as small actions of hyperbolic groups.

Proof of Corollary III.3. Every very small F -action on T is a limit of free simplicial F -actions on R-trees T_{i} by [BF]. This limit may be understood as convergence of length functions, or equivalently [Pa 2] as convergence in the equivariant Gromov-Hausdorff topology: given Q ∈ T, a finite subset F_{0} ⊂ F, and ε > 0, then for i large there exists Q_{i} ∈ T_{i} such that the distance between Q_{i} and gQ_{i} in T_{i} is ε-close to d(Q, gQ) for g ∈ F_{0}. It follows easily that Lemma III.2 is also valid for very small actions.

Now let T be an R-tree with a minimal action of F satisfying (BBT1). A ray ρ ⊂ T, with origin a point Q ∈ T, is the image of an isometric map (0, ∞) → T, with closure ρ̄ = ρ ∪ {Q}. Two rays (with different origins) are equivalent if their intersection has infinite length. We denote δT the set of equivalence classes. The action of F on T induces a natural action of F on δT.

Suppose ρ is a ray, R, S are points of T, and v_{n} ∈ F is a sequence such that the length of [R, v_{n}S] ∩ ρ goes to infinity as n → ∞. Then clearly the length of [R′, v_{n}S′] ∩ ρ′ goes to infinity for any R′, S′ ∈ T and ρ′ equivalent to ρ.

Lemma III.4. Suppose the length of [R, v_{n}S] ∩ ρ goes to infinity as n → ∞. Then v_{n} converges to some X ∈ δF depending only on ρ. Furthermore the length of [R, w_{n}S] ∩ ρ goes to infinity for every sequence w_{n} → X.

Proof. We may assume R = S = Q, the origin of ρ. First note that |v_{n}| goes to infinity with n. Thus some subsequence converges to a point of δF. To prove uniqueness of the limit, we suppose v_{n}, v′_{n} are two sequences as in the lemma, converging to X, X′ ∈ δF respectively, and we show X = X′. Indeed, with the notations of Part I, the

distance between Q and (v_{n} ∧ v′_{n})Q goes to infinity as n → ∞ by (BBT2). This implies that |v_{n} ∧ v′_{n}| goes to infinity, i.e. X = X′.

Denoting X_{t(p)} = v_{p} ∧ X = lim_{i→∞}(v_{p} ∧ v_{i}), we note that the length of [Q, X_{t(p)}Q] ∩ ρ goes to infinity with p by (BBT2). Property (BBT1) then implies that the length of [Q, w_{n}Q] ∩ ρ goes to infinity for any w_{n} → X.

Given R, S ∈ T and a ray ρ, minimality of the action of F on T provides a sequence v_{n} as in Lemma III.4. We define j(ρ) ∈ δF as the limit X of v_{n}. Of course j(ρ) depends only on the equivalence class of ρ.

Lemma III.5. This defines an F -equivariant injection j : δT → δF. If H is a homothety of T as in Theorem II.1, then α(j(ρ)) = j(H(ρ)) for any ray ρ. The image of j is disjoint from every δ(Stab Q).

Proof. Equivariance and injectivity are clear. For the second assertion, simply note that

H([R, v_{n}S] ∩ ρ) = [R′, α(v_{n})S′] ∩ H(ρ)

with R′ = H(R) and S′ = H(S). Since the length of [Q, v_{n}Q]∩ρ is 0 if v_{n} ∈ Stab Q, we have j(ρ) ∉ δ(Stab Q).

IV. Analyzing the tree

Let α be an automorphism of F with positive index

ind (α) = rk Fix α + \frac{1}{2}a(α) − 1 > 0,

and let T, H be given by Theorem II.1. Recall that H has at least one fixed point.

Lemma IV.1.

(1) If Q is fixed by H, then Stab Q is α-invariant. We denote α^{Q} = α|_{Stab Q}.

(2) If Q is the only fixed point of H, then Stab Q contains Fix α (i.e. Fix α = Fix α^{Q}).

Proof. (1) From wQ = Q it follows α(w)Q = α(w)HQ = HwQ = Q.

(2) From α(w) = w it follows HwQ = α(w)HQ = wQ, so that wQ = Q if H has only one fixed point.

Our main goal will be to relate ind (α) to data coming from T and automorphisms of groups of smaller rank (such as Stab Q, which has rank < n by Theorem 2). In particular, we will map subsets of A(α) injectively into A(α^{Q}) thanks to the following lemma (recall that A(α) is the set of equivalence classes of attracting fixed points of α).

Lemma IV.2. Let Q ∈ T be fixed by H. Let A_{Stab Q}(α) ⊂ A(α) be the set of equivalence classes of attracting fixed points having nonempty intersection with δ(Stab Q).

(1) There is an injection θ_{Q} : A_{Stab Q}(α) ↪ A(α^{Q}).

(2) If Q is the only fixed point of H, then θ_{Q} is a bijection.

Proof. (1) Given a class in A_{Stab Q}(α), represent it by some X ∈ δ(Stab Q). Proposition I.1 implies that X is an attracting fixed point of α^{Q}, and we map the class of X in A(α) to the class of X in A(α^{Q}). Since fixed points X, X′ ∈ δ(Stab Q) that are inequivalent for α are inequivalent for α^{Q}, we get the required (non-canonical) injection.

(2) If Q is the only fixed point of H, then Fix α = Fix α^{Q} ⊂ Stab Q. This implies that any X′ equivalent to X ∈ δ(Stab Q) also belongs to δ(Stab Q) and is equivalent to X as a fixed point of α^{Q}. It follows that θ_{Q} is a bijection: the class of an attracting X ∈ δ(Stab Q) is the image of the class of X in A_{Stab Q}(α) (note that X is attracting for α by Proposition I.1).

Now we distinguish several cases.

Case A: λ > 1.

In this case the map H has a unique fixed point Q. Note that T has BBT by Corollary III.3. We will use properties (BBT1) and (BBT2) relative to Q.

If a, b ∈ T, we say that the segments [Q, a] and [Q, b] overlap if their intersection is a (non-degenerate) arc. This is equivalent to saying that a and b belong to the same component of T \ {Q}.

If H sends a component A of T \ {Q} to itself, then A contains a unique Hinvariant ray ρ at Q , i.e. ρ(λx) = Hρ(x) for x > 0 (see [Li]). Such rays will be called eigenrays of H. Notice that \overline{ρ} contains [Q, a] ∩ [Q, Ha] for every a ∈ A. (As noted in [Li], the restriction of H to A is topologically conjugate to a hyperbolic isometry of an R-tree, and ρ corresponds to the translation axis).

Proposition IV.3. If X ∈ δF is attracting, then either X ∈ δ(Stab Q), or there exists an eigenray ρ of H such that X = j(ρ).

Remark. The conclusions are mutually exclusive by Lemma III.5.

Proof. We shall use repeatedly equations such as H^{p}(X_{i}Q) = α^{p}(X_{i})Q, obtained by combining α(w)H = Hw and HQ = Q.

Let X = x_{1} ⋯ x_{i} ⋯ be an attracting fixed word (in a given free basis g_{1}, …, g_{n} of F). If X_{i}Q = Q for infinitely many i, we have X ∈ δ(Stab Q). Otherwise we fix an integer i with X_{i}Q ≠ Q such that the sequence c_{X}(α^{p}X_{i}) is strictly increasing (see (∗) in Part I).

Recall (Theorem 2) that the action of Stab Q on π_{0}(T \ {Q}) has finitely many orbits. Thus there exist d ≥ 1 and w ∈ Stab Q such that [Q, wX_{i}Q] and [Q, H^{d}(X_{i}Q)] overlap. We distinguish two cases, depending on whether w is trivial or not.

• First assume that w is trivial. Let A be the component of T \ {Q} containing X_{i}Q, and ρ the corresponding eigenray of H^{d}.

By assumption the sequence θ(p) = c_{X}(α^{pd}X_{i}) is strictly increasing. In particular notice X_{θ(p)} = (α^{pd}X_{i}) ∧ (α^{(p+1)d}X_{i}). By (BBT2), the distance from X_{θ(p)}Q to [(α^{pd}X_{i})Q, (α^{(p+1)d}X_{i})Q] = [H^{pd}(X_{i}Q), H^{(p+1)d}(X_{i}Q)] is bounded by C. On the other hand the intersection between [Q, H^{pd}(X_{i}Q)] and [Q, H^{(p+1)d}(X_{i}Q)] = [Q, H^{d}(H^{pd}(X_{i}Q))] is contained in the H^{d}-eigenray \overline{ρ}. Furthermore the length of this intersection is a constant multiple of λ^{pd}. This implies that the length of [Q, X_{θ(p)}Q] ∩ ρ goes to infinity with p, i.e. X = j(ρ) (see Lemma III.4).

There remains to show that ρ is invariant under H, not only under H^{d}. This follows from Lemma III.5 since j(ρ) = α(j(ρ)) = j(H(ρ)) and hence ρ = H(ρ).

• Now we assume that the element w ∈ Stab Q is not trivial. In this case we show X ∈ δ(Stab Q).

The length of the intersection between [Q, α^{p}(wX_{i})Q] = [Q, H^{p}(wX_{i}Q)] and [Q, (α^{p+d}(X_{i}))Q] = [Q, H^{p}(H^{d}(X_{i}Q))] goes to infinity as p goes to ∞. By (BBT2) this implies that d(Q, w_{p}Q) goes to infinity, with w_{p} = α^{p}(wX_{i}) ∧ α^{p+d}(X_{i}). This in turn implies that the length of the word w_{p} goes to infinity.

Write w_{p} = α^{p}(w)α^{p}(X_{i})∧α^{p+d}(X_{i}) and recall that c_{X}(α^{p}(X_{i})) goes to infinity with p. If the amount of cancellation between α^{p}(w) and α^{p}(X_{i}) does not remain bounded, then X is a limit point of the sequence α^{p}(w^{−1}) and belongs to δ(Stab Q). Similarly, if the length of α^{p}(w) ∧ α^{p+d}(X_{i}) is unbounded, then X is a limit point of α^{p}(w) and belongs to δ(Stab Q).

Since |w_{p}| goes to infinity, the only remaining possibility is that |α^{p}(w)| is bounded. Choose a subsequence p_{k} such that α^{p_{k}}(w) is a nontrivial word v ∈ Stab Q independent of k. We then get vX = X since α^{p_{k}}(X_{i}) and α^{p_{k}+d}(X_{i}) converge to X as p goes to ∞. This implies X ∈ δ(Stab Q) since X is a limit point of one of the sequences v^{p} or v^{−p}.

Let V(Q) be the set of orbits of the action of Stab Q on components of T \ Q, and let V^{H}(Q) ⊂ V(Q) be the subset consisting of orbits containing a component fixed by H.

Proposition IV.4.

(1) There exists an injection τ : A(α) ↪ A(α^{Q}) ∪ V(Q) (disjoint union).

(2) The image of τ is A(α^{Q}) ∪ V^{H}(Q).

Remark. Assertion (2) is not needed for the proof of Theorem 1.

Recalling that Fix α = Fix α^{Q} and that ind (α) = rk Fix α + \frac{1}{2}a(α) − 1, we get from Assertion (1):

Corollary. ind (α) ≤ ind (α^{Q}) + \frac{1}{2}v(Q).

Proof of Proposition IV.4. (1) If an attracting fixed point X of α belongs to δ(Stab Q), so does every X′ equivalent to X since Stab Q contains Fix α. We map the corresponding set of classes A_{Stab Q}(α) bijectively onto A(α^{Q}) using Lemma IV.2.

If an attracting fixed point X does not belong to δ(Stab Q), we map the class of X to the orbit under Stab Q of the component A_{X} of T \ {Q} containing the points X_{p}Q for p large and the associated eigenray ρ = ρ_{X}. To prove that τ is well-defined and injective on A(α) \ A_{Stab Q}(α), we need to check that A_{X} and A_{X′} belong to the

same Stab Q-orbit if and only if X and X′ are equivalent.

If X and X′ are equivalent, then by definition there exists w ∈ Fix α with X′ = wX. Equivariance of j (see Lemma III.5) gives j(wρ_{X}) = wj(ρ_{X}) = X′ = j(ρ_{X′}) and thus wρ_{X} = ρ_{X′}. In particular wA_{X} = A_{X′}.

Conversely, assume that some h ∈ Stab Q maps A_{X} to A_{X′}. The intersection of the rays ρ_{X} and h^{−1}ρ_{X′} is a nondegenerate segment (Q, Q′]. For a ∈ (Q, Q′] close enough to Q we have Ha ∈ (Q, Q′] and therefore hHa = Hha. But we also have Hha = α(h)Ha. This implies that h^{−1}α(h) fixes the nondegenerate subsegment [Q, Ha] of [Q, Q′]. As by Theorem II.1 the group action on T has trivial arc stabilizers, it follows that α(h) = h. In particular the isometry defined by h commutes with H and therefore the fact that ρ_{X} and h^{−1}ρ_{X′} overlap nontrivially implies ρ_{X} = h^{−1}ρ_{X′}. We get X = j(ρ_{X}) = j(h^{−1}ρ_{X′}) = h^{−1}X′, and hence X and X′ are equivalent.

(2) To prove Assertion (2), it suffices to consider an H-invariant component A of T \ Q, and to show that its orbit

under Stab Q belongs to the image of τ. Let ρ be the H-eigenray contained in A, and X = j(ρ) ∈ δF. We now show that X is an attracting fixed point of α. This will complete the proof since the class of X maps to the Stab Q-orbit of A.

From Lemma III.5 it follows α(X) = α(j(ρ)) = j(H(ρ)) = j(ρ) = X. By way of contradiction, assume that X is not attracting. Writing α(X_{i}) = X_{k(i)}Z_{i} with |Z_{i}| ≤ B as in Part I, there exist arbitrarily large i such that k(i) ≤ i + p for some fixed integer p. For these values of i, we claim that d(α(X_{i})Q, [Q, X_{i}Q]) is bounded by a constant independent of i. Indeed

d(α(X_{i})Q, X_{k(i)−p}Q) ≤ (B + p) max_{j} d(Q, g_{j}Q),

and d(X_{k(i)−p}Q, [Q, X_{i}Q]) ≤ C by (BBT1) since k(i) − p ≤ i.

On the other hand [Q, α(X_{i})Q] ∩ ρ = [Q, H(X_{i}Q)] ∩ ρ = H([Q, X_{i}Q] ∩ ρ) has length λ times bigger than [Q, X_{i}Q] ∩ ρ. As the length of [Q, X_{i}Q] ∩ ρ goes to infinity as i → ∞, this contradicts the above observation.

For later purposes we also need the following fact:

Lemma IV.5. Let A, A′ be components of T \ {Q} belonging to the same Stab Qorbit. Suppose A is invariant under H and A′ is invariant under mH for some m ∈ Stab Q. Then there exists c ∈ Stab Q such that m = cα(c^{−1}) (hence i_{m} ∘ α is similar to α).

Proof. Let c be an element of Stab Q mapping A to A′. For a ∈ A close enough to Q on the H-invariant ray we write cHa = (mH)ca = mα(c)Ha. Thus c^{−1}mα(c) fixes a nondegenerate segment. Since arc stabilizers are trivial this implies m = cα(c^{−1}).

Case B: λ = 1.

If λ = 1, then T is simplicial. In this case we choose T so as to minimize the number of edges of the quotient graph Γ = T/F.

Lemma IV.6. If H has more than one fixed point, then Γ = T/F has only one edge.

Proof. Let e be an edge fixed (pointwise) by H and let T′ be the tree obtained by collapsing each component of the orbit of e to a point. This orbit is preserved by H, so that H induces an isometry H′ of T′ satisfying the commutation equation α(w)H′ = H′w with the induced F -action on T′. If Γ has more than one edge, the action of F on T′ is nontrivial. This contradicts the choice of T.

Hence for λ = 1 we need only consider three more cases:

(1) H has exactly one fixed point Q.

(2) H has more than one fixed point and Γ is a segment.

(3) H has more than one fixed point and Γ is a loop.

Case B1: H has exactly one fixed point Q.

Proposition IV.7. There is an injection τ : A(α) ↪ A(α^{Q}).

Since Fix α = Fix α^{Q} by Lemma IV.1 we get:

Corollary. ind (α) ≤ ind (α^{Q}).

Proof of Proposition IV.7. By Lemma IV.2, it suffices to prove that every attracting fixed point of α belongs to δ(Stab Q). Recall that the action of F on T is simplicial with trivial edge stabilizers. Bass-Serre theory expresses F as a free product whose factors are vertex groups and π_{1}Γ (see [Ser, I.5.1]). We use this to get a preferred free basis of F as follows.

Let T_{0} ⊂ T be a finite subtree containing Q and projecting isomorphically onto a maximal subtree Γ_{0} ⊂ Γ. Orient the edges of Γ \ Γ_{0}. The basis we construct consists of the element of F naturally associated to each edge of Γ \ Γ_{0}, together with bases of the groups Stab V, for V a vertex of T_{0}.

Such a basis has the following property. Suppose w ∈ F \Stab Q, and w′ contains w as an initial subword. Then the segments [Q, wQ] and [Q, w′Q] intersect in a nondegenerate segment.

Now let X be an attracting fixed word of α (in a preferred basis of F). We show X ∈ δ(Stab Q). Fix an integer p. Since H is an isometry having a fixed point, the midpoint of the segment between X_{p}Q and H(X_{p}Q) = α(X_{p})Q is equal to Q because it is fixed by H (see e.g. [MoSh, Lemma II.2.16]). The segments [Q, X_{p}Q] and [Q, α(X_{p})Q] thus do not overlap. For p large, the word X_{p} is an initial subword of α(X_{p}). Our choice for the basis of F then implies X_{p} ∈ Stab Q. Hence X ∈ δ(Stab Q) as required.

Case B2: H has more than one fixed point and Γ is a segment.

Let e = [Q, R] be an edge of T fixed (pointwise) by H. Bass-Serre theory gives a nontrivial decomposition F = Stab Q ∗ Stab R. This decomposition is α-invariant by Lemma IV.1.

Proposition IV.8.

(1) Fix α = Fix α^{Q} ∗ Fix α^{R}.

(2) There is an injection τ : A(α) ↪ A(α^{Q}) ∪ A(α^{R}).

Corollary. ind (α) ≤ ind (α^{Q}) + ind (α^{R}) + 1.

Proof of Proposition IV.8. The proof we give is purely algebraic, using only the α-invariant decomposition of F. We choose a free basis of F consisting of a basis of Stab Q together with a basis of Stab R.

Write any finite word v ∈ F as a product of subwords belonging alternatively to both factors. If v is fixed by α, then each subword has to be fixed. This shows the first assertion.

Now suppose X is any attracting fixed infinite word. If X contains infinitely many letters from both factors, then it is an infinite product of subwords belonging alternatively to Fix α^{Q} and Fix α^{R}, and X ∈ δ(Fix α). This is impossible since X is attracting. Thus X is equivalent to an infinite word contained in one of the factors, and we get τ using Lemma IV.2.

Case B3: H has more than one fixed point and Γ is a loop.

Let again e = [Q, R] be an edge of T fixed (pointwise) by H. We now have (by Bass-Serre theory) F = (Stab Q)∗ < t >, where t is any element such that t(Q) = R. Note that α(t)Q = α(t)HQ = HtQ = R, so that α(t) = tu with u ∈ Stab Q.

If there exists t′ with t′(Q) = R and α(t′) = t′, we have an α-invariant decomposition as before and the previous analysis yields ind (α) ≤ ind (α^{Q}) + 1. We assume therefore that there is no such t′. This implies that u cannot be written u = vα(v^{−1}) with v ∈ Stab Q, since otherwise we can set t′ = tv.

Proposition IV.9.

(1) Fix α = Fix α^{Q} ∗ t Fix (i_{u} ∘ α^{Q}) t^{−1}.

(2) There is an injection τ : A(α) ↪ A(α^{Q}) ∪ A(i_{u} ∘ α^{Q}).

Corollary. ind (α) ≤ ind (α^{Q}) + ind (i_{u} ∘ α^{Q}) + 1.

Proof of Proposition IV.9. Choose a free basis of F consisting of t together with a basis of Stab Q. Any element of F has a unique reduced expression

w = v_{0}t^{ε_{1}}v_{1}t^{ε_{2}} … v_{p−1}t^{ε_{p}}v_{p}

where ε_{i} = ±1 and v_{i} is a (possibly trivial) word not containing t^{±1}. We study the word α(w), paying special attention to the letters t^{±1}.

Recall that α(v_{i}) does not contain t^{±1} and that α(t) = tu. Thus no new letters t^{±1} appear in α(w). Also note that there can be no cancellation between the p letters t^{ε_{i}} in α(w), since the image of a subword tvt^{−1} is tuα(v)u^{−1}t^{−1} (similarly α(t^{−1}vt) = u^{−1}t^{−1}α(v)tu).

Now assume that w is fixed by α. This forces the words v_{i} (1 ≤ i ≤ p − 1) to satisfy equations, whose form depends on ε_{i} and ε_{i+1}:

v_{i} = α(v_{i}) if w contains t^{−1}v_{i}t
v_{i} = uα(v_{i})u^{−1} if w contains tv_{i}t^{−1}
v_{i} = uα(v_{i}) if w contains tv_{i}t
v_{i} = α(v_{i})u^{−1} if w contains t^{−1}v_{i}t^{−1}.

The first two equations express that v_{i} should belong to Fix (α^{Q}) or Fix (i_{u}∘α^{Q}). On the other hand the assumption that u cannot be written u = vα(v^{−1}) prevents the other two equations from being satisfied. Thus the letters t and t^{−1} alternate in w. Furthermore the relation α(w) = w also implies v_{0} ∈ Fix α^{Q}, ε_{1} = 1, and v_{p} ∈ Fix α^{Q}, ε_{p} = −1.

Summing up, the words invariant under α are precisely those of the form v_{0}tv_{1}t^{−1}v_{2}t … v_{2q} with v_{i} ∈ Fix α^{Q} for i even and v_{i} ∈ Fix (i_{u} ∘ α^{Q}) for i odd. We have shown Assertion 1.

Proving Assertion 2 is now easy. If an attracting fixed word X contains t^{±1} infinitely often, then X is an infinite product v_{0}tv_{1}t^{−1}v_{2}tv_{3}t^{−1} ⋯ with the v_{i}’s as above. This implies X ∈ δ(Fix α), a contradiction. Thus X is equivalent to a word X_{0} not

containing t^{±1}, or to a word tX_{0} with no letter t^{±1} in X_{0}. The word X_{0} is an attracting fixed word of α^{Q} in the first case, of i_{u} ∘ α^{Q} in the second: if α(tX_{0}) = tX_{0}, then tuα^{Q}(X_{0}) = tX_{0} and therefore X_{0} = uα^{Q}(X_{0}) = (i_{u} ∘ α^{Q})(X_{0}).

Once again we obtain τ using Lemma IV.2.

Notice that results analogous to Propositions IV.8 and IV.9 are true if T has more than one orbit of edges, provided H fixes each of them.

V. The induction

Let α_{0}, …, α_{k} be automorphisms of F representing the same outer automorphism and belonging to distinct similarity classes. We prove the following inequality by induction on n = rk F:

∑_{i=0}^{k} ind (α_{i}) ≤ n − 1.

It is clear if n = 1, so we assume n ≥ 2. We also assume ind (α_{i}) > 0 ∀i. Since the index is a similarity invariant, we will be free to replace each α_{i} by a similar automorphism when needed.

Apply Theorem II.1 to α = α_{0}. We get a tree T and a homothety H : T → T with stretching factor λ. If β = i_{m} ∘ α with m ∈ F, we associate to β the map H_{β} = mH. It satisfies β(w)H_{β} = H_{β}w for all w ∈ F (see Remark II.2). If β = i_{c} ∘α∘(i_{c})^{−1} = i_{cα(c^{−1})} ∘α is similar to α, we get H_{β} = cα(c^{−1})H_{α} = cH_{α}c^{−1}. In particular the fixed point sets satisfy Fix H_{β} = cFix H_{α}.

For simplicity we write H_{i} for H_{α_{i}}. We may assume as before that the quotient graph Γ = T/F has only one edge if λ = 1 and some H_{i} has more than one fixed point (see Remark II.2 and Lemma IV.6).

By Assertion 3 of Theorem II.1, each H_{i} has at least one fixed point. If Q is fixed by H_{i}, then Stab Q is α_{i}-invariant (Lemma IV.1). We denote α_{i}^{Q} the induced automorphism.

Lemma V.1. Suppose Q ∈ T is fixed by both H_{i} and H_{j} (i ≠ j) and rk Stab Q ≥ 2. Then α_{i}^{Q}, α_{j}^{Q} represent the same outer automorphism of Stab Q and belong to distinct similarity classes in Aut (Stab Q).

Proof. If α_{j} = i_{h} ∘ α_{i}, we have h ∈ Stab Q because H_{j} = hH_{i}. Thus α_{i}^{Q} and α_{j}^{Q} represent the same outer automorphism of Stab Q.

Now suppose there exists v ∈ Stab Q such that α_{j}(g) = vα_{i}(v^{−1}gv)v^{−1} for all g ∈ Stab Q. Then

hα_{i}(g)h^{−1} = α_{j}(g) = vα_{i}(v^{−1})α_{i}(g)α_{i}(v)v^{−1}

for g ∈ Stab Q. Since Stab Q has rank ≥ 2, we deduce h = vα_{i}(v^{−1}), so that α_{j}(g) = vα_{i}(v^{−1}gv)v^{−1} holds for every g ∈ F. This is a contradiction since α_{i} and α_{j} are not similar.

As in Part IV, we now distinguish several cases.

Case A: λ > 1.

Each H_{i} has exactly one fixed point Q_{i}. Recall that Fix H_{β} = cFix H_{α} if β = i_{c} ∘ α ∘ (i_{c})^{−1} is similar to α. Replacing each α_{i} by a similar automorphism, we may then assume that for i ≠ j either Q_{i} = Q_{j}, or Q_{i} and Q_{j} belong to different F -orbits. Let Q ⊂ T be the set of all points Q_{i}, and π : {0, …, k} → Q the map taking i to Q_{i}.

Proposition IV.4 yields injections τ_{i} : A(α_{i}) → A(α_{i}^{Q_{i}}) ∪ V(Q_{i}). We denote v_{i}(Q_{i}) the cardinality of τ_{i}(A(α_{i}))∩V(Q_{i}). Using Proposition IV.4 and its corollary we write

∑_{i=0}^{k} ind (α_{i}) ≤ ∑_{i=0}^{k}(ind (α_{i}^{Q_{i}}) + \frac{1}{2}v_{i}(Q_{i})) = ∑_{Q∈Q} ∑_{i∈π^{−1}(Q)}(ind (α_{i}^{Q_{i}}) + \frac{1}{2}v_{i}(Q_{i})).

We then have

∑_{i∈π^{−1}(Q)} ind (α_{i}^{Q_{i}}) ≤ rk Stab Q − 1.

This is clear for points Q with rk Stab Q ≤ 1. For other points it follows from Lemma V.1 and the induction hypothesis, as rk Stab Q ≤ n − 1 by Theorem 2.

On the other hand

∑_{i∈π^{−1}(Q)} \frac{1}{2}v_{i}(Q_{i}) ≤ \frac{1}{2}v(Q)

because by Lemma IV.5 two components of T \{Q} containing rays invariant under H_{i} and H_{j} respectively cannot be in the same Stab Q-orbit if i ≠ j.

As a result we obtain

∑_{i=0}^{k} ind (α_{i}) ≤ ∑_{Q∈Q} ∑_{i∈π^{−1}(Q)}(ind (α_{i}^{Q_{i}}) + \frac{1}{2}v_{i}(Q_{i}))

≤ ∑_{Q∈Q}(rk Stab Q − 1 + \frac{1}{2}v(Q))

≤ n − 1.

The third inequality follows from Theorem 2 since different points of Q are in different F -orbits.

Case B1: λ = 1 and each H_{i} has exactly one fixed point.

The proof is the same, using the corollary to Proposition IV.7. One does not need the v terms.

Now we suppose some H_{i} (say H_{0}) fixes (pointwise) an edge e = [Q, R]. Recall that we have assumed the graph Γ = T/F has only one edge.

Case B2: H_{0} has more than one fixed point, and Γ is a segment.

Note that for i > 0 the map H_{i} has only one fixed point Q_{i}. Otherwise we could replace α_{i} by a similar automorphism and get H_{i} to fix e (recall that there is only one F -orbit of edges). This would contradict triviality of edge stabilizers since H_{i} = m_{i}H for some nontrivial m_{i} ∈ F.

The fixed point Q_{i} cannot be the midpoint of an edge because Stab Q_{i} would be trivial (F acts without inversions) and Proposition IV.7 would imply ind (α_{i}) < 0. Thus Q_{i} belongs to the orbit of either Q or R. Changing α_{i} within its similarity class if needed, we may assume Q_{i} = Q or R.

Taking π to be the obvious map from {1, …, k} to {Q, R}, we get by the corollaries to Propositions IV.8 and IV.7

∑_{i=0}^{k} ind (α_{i}) ≤ ind (α_{0}^{Q}) + ind (α_{0}^{R}) + 1 + ∑_{i∈π^{−1}(Q)} ind (α_{i}^{Q}) + ∑_{i∈π^{−1}(R)} ind (α_{i}^{R})

≤ 1 + (ind (α_{0}^{Q}) + ∑_{i∈π^{−1}(Q)} ind (α_{i}^{Q})) + (ind (α_{0}^{R}) + ∑_{i∈π^{−1}(R)} ind (α_{i}^{R}))

≤ 1 + (rk Stab Q − 1) + (rk Stab R − 1)

≤ n − 1.

The third inequality comes from Lemma V.1 and the induction hypothesis, as F is isomorphic to the nontrivial free product Stab Q ∗ Stab R (see the discussion in Part IV).

Case B3: H_{0} has more than one fixed point, and Γ is a loop.

Arguing as in the beginning of case B2, we may assume that H_{0} fixes an edge e = [Q, R] and that Q is the only fixed point of H_{i} for i > 0 (now there is only one F -orbit of vertices).

With the notations of Part IV we have F = (Stab Q)∗ < t > with α_{0}(t) = tu, u ∈ Stab Q. If t may be chosen with α_{0}(t) = t, we simply write

∑_{i=0}^{k} ind (α_{i}) ≤ ind (α_{0}^{Q}) + 1 + ∑_{i=1}^{k} ind (α_{i}^{Q}) ≤ rk Stab Q = n − 1.

Otherwise we recall that u cannot be written u = vα_{0}(v^{−1}) with v ∈ Stab Q and we apply the corollaries to Propositions IV.7 and IV.9. We get

∑_{i=0}^{k} ind (α_{i}) ≤ ∑_{i=0}^{k} ind (α_{i}^{Q}) + ind (i_{u} ∘ α_{0}^{Q}) + 1.

There is nothing to prove if Stab Q has rank 1. Otherwise we argue as follows. By Lemma V.1, the automorphisms α_{0}^{Q}, α_{1}^{Q}, …, α_{k}^{Q}, i_{u} ∘ α_{0}^{Q} represent the same outer automorphism of Stab Q. If we show that no two of them are similar, the inductive proof will be complete since we can write

∑_{i=0}^{k} ind (α_{i}) ≤ rk Stab Q − 1 + 1 = n − 1.

By Lemma V.1 we need only check that i_{u} ∘ α_{0}^{Q} is not similar to any of the others. Arguing as in the proof of Lemma V.1 we first see that it is not similar to α_{0}^{Q} since

i_{u} ∘ α_{0}^{Q} = i_{v} ∘ α_{0}^{Q} ∘ (i_{v})^{−1} ⟹ u = vα_{0}^{Q}(v^{−1}).

Then we note that i_{u} ∘ α_{0} is similar to α_{0} in Aut (F) since i_{u} ∘ α_{0} = (i_{t})^{−1} ∘ α_{0} ∘ i_{t}. It follows that i_{u} ∘ α_{0} and α_{i} are not similar for i ≥ 1. By Lemma V.1, i_{u} ∘ α_{0}^{Q} and α_{i}^{Q} are not similar in Aut (Stab Q).

VI. Discussion and remarks

In this last section we want to discuss some properties of certain classes of automorphisms of F, with respect to their index. We will not give formal proofs but rather indicate the reason for our claims; details may be worked out elsewhere.

First we recall the definition of the index of an outer automorphism of the free group α̂ ∈ Out (F), induced by α ∈ Aut (F). Let S(α̂) denote the set of similarity classes [α′] of automorphisms α′ inducing the outer automorphism α̂′ = α̂. We define

ind (α̂) := ∑_{[α′]∈S(α̂)} max(ind (α′), 0).

Our main result, Theorem 1’, is equivalent to the inequality

ind (α̂) ≤ rk (F) − 1  (1)

for all α̂ ∈ Out (F).

Next we observe that the techniques for computing a basis of the fixed subgroup as well as representatives for each equivalence class of attracting fixed points, described in [CL 1] for positive automorphisms, can be extended to all automorphisms of F. In [CL 1] the only specific property of positive automorphisms used is the fact that for any w ∈ F there is an obvious lower bound for the length of α^{k}(w) for any k ≥ 1. A similar bound can be deduced for arbitrary α ∈ Aut (F) if we use a relative train track representative for α as constructed in [BH]. This gives the following, which can be used to verify some of the statements below:

VI.1. There is a combinatorial algorithm which computes for any α ∈ Aut (F) the indices ind (α) and ind (α̂).

There is an obvious question as to when the inequality (1) is an equality. Using Theorem 1’ together with standard Nielsen-Thurston theory one can compute the index of any geometric automorphism, i.e. an automorphism which is induced by a homeomorphism of a surface M with boundary, where π_{1}M ≅ F. For example, if α̂ is induced by a pseudo-Anosov automorphism which fixes each separatrix at every singularity of the stable foliation, then ind (α̂) is indeed equal to χ(S) = rk (π_{1}S) − 1. These arguments extend to the case where α is a multiple Dehn twist on disjoint closed curves, possibly with pseudo-Anosov components on the complementary subsurfaces. This shows:

VI.2. Every geometric automorphism α̂ ∈ Out (F) has a power α̂^{k} with

ind (α̂^{k}) = rk (F) − 1.

This fact was used in earlier work of the fourth author to describe new classes of non-geometric automorphisms with properties very similar to those of geometric ones. Also, from the above discussion it is easy to construct geometric α’s which do not satisfy equality in (1).

Dehn twists on surfaces have been generalized in [CL 2] to Dehn twist automorphisms of free groups. It is shown in [CL 3] that every Dehn twist automorphism D̂ ∈ Out (F) has maximal index ind (D̂) = rk (F) − 1. This implies (see [CL 3]):

VI.3. Every linear growth automorphism α̂ ∈ Out (F) has a power α̂^{k} with

ind (α̂^{k}) = rk (F) − 1.

Recall that α̂ is said to have polynomial growth of degree ≤ k if, for every g ∈ F, there exists C such that α^{t}(g) is conjugate to an element of length ≤ Cn^{k} for every t ≥ 1 (for some, hence any, α representing α̂).

We now want to discuss how or why the index of an outer automorphism can be non-maximal. Our proof of Theorem 1’ gives rather explicit information about this, and we can distinguish between two qualitatively different phenomena.

VI.4. Given two automorphisms α ∈ Aut (F′) and β ∈

Aut (F”), wecancomposethemcanonicallytoanewautomorphismα∗β ∈ Aut (F′∗ F′′). On the level of outer automorphisms, however, there is no such canonical composition. Given α̂ and β̂, one first has to choose representatives α′ and β′, and then one can define a freely composed outer automorphism { \atop α′ ∗ β′}. If one has both ind (α′) ≥ 0 and ind (β′) ≥ 0, then one obtains

ind ({ \atop α′ ∗ β′}) = ind (α̂′) + ind (β̂′) + 1 ,  (2)

which gives maximal index for { \atop α′ ∗ β′} if both α̂ and β̂ have maximal index. If, however, the index of α′ or β′ is negative, then the index of { \atop α′ ∗ β′} differs from the maximal value (2) by \frac{1}{2} or 1.

Similarly, for α ∈ Aut (F), the HNN-composition α_{+} ∈

Aut (F*⟨t⟩) given by α_{+}(w) = vα(w)v^{−1} if w ∈ F, α_{+}(t) = tu (v, u ∈ F), which has been considered in case B3 of Parts IV and V, “connects” similarity classes of α̂ with varying possibilities for their index (according to the choice of v and u). Again, the index of α̂_{+} ∈ Out (F ∗ ⟨t⟩) can drop by \frac{1}{2} or 1 from the maximal value

ind (α̂_{+}) = ind (α) + 1 .

We obtain as immediate consequence the existence of outer automorphisms with all positive powers of very small index: for example, for α : a → a, b → ba, c_{k} → b^{k}c_{k}b^{k} (k = 1, …, m) one has rk (F) = m + 2 and ind (α̂^{t}) = 1 for all t ≥ 1.

VI.5. It follows from [BH] that α̂ has exponential growth if for some stratum of some (and hence any) relative train track representative of α̂ the transition matrix has Perron-Frobenius eigenvalue strictly bigger than 1. Otherwise, after possibly refining the strata structure, every transition matrix is either 0 or a permutation matrix. In this case α̂ has polynomial growth, and it is not hard to see that a suitable power of α̂ arises precisely as iterated free composition or HNN-composition as in VI.4, where all ???? starting ???? factors are Dehn twist automorphisms. One can show that at each composition the degree of polynomial growth can be raised by one, but only at the expense of strictly decreasing the index of the composed automorphism with respect to the maximal possible composition value. This shows:

Every polynomially but non-linearily growing (and hence non-geometric) outer automorphism α̂ has non-maximal index:

ind (α̂) ≤ rk (F) − \frac{3}{2}

Let us now turn to the second basic phenomenon which produces non-maximal index of an outer automorphism.

If α “commutes” with a homothety H of an R-tree T as in Theorem II.1, then by the analysis of Part V the index of α̂ is bounded by the index ind (T) of the tree T, defined as the maximal value of the sum ∑_{ℓ=1}^{q}(rk Stab Q_{ℓ} + \frac{1}{2}v(Q_{ℓ}) − 1) that appears in Theorem 2 (see [GL], which considers i(T) = 2 ind (T)). In [GL] a distinction is drawn between the case when the action of F on T is geometric, in which case the index of T is shown to be rk (F) − 1 (i.e. maximal), and the case when T is non-geometric, where one proves \frac{1}{2} ≤ ind (T) ≤ rk (F) − \frac{3}{2}.

A prototype of geometric actions is given by pseudo-Anosov automorphisms and the R-tree dual to an invariant foliation. However, somehow surprisingly, there are also non-geometric automorphisms for which the invariant R-tree is geometric but not simplicial. These parageometric automorphisms can be found even among irreducible automorphisms with irreducible powers (i.e. no α^{t} with t ≥ 1 leaves a proper free factor of F invariant), see [BF] or [Le], and those deserve special attention as they seem rather atypical among all irreducible ones. It is shown in [BF] that precisely the non-geometric and non-parageometric irreducible automorphisms admit train track representatives without non-trivial Nielsen paths. Combining their result with ours can be summarized as follows:

VI.6. Assume that the outer automorphism α̂ ∈ Out (F) is irreducible, let f: τ → τ be a train track representative for α with the least number of indivisible Nielsen paths among all such representatives, and let T be the invariant tree obtained from iterating f as in Part II. Then precisely one of the following two cases occurs:

(i) The index of α̂ satisfies

\frac{1}{2} ≤ ind (α̂^{t}) ≤ rk (F) − \frac{3}{2}

for all t ≥ 1. The action of F on T is non-geometric. The train track τ has no non-trivial Nielsen path with respect to f or to any f^{t}.

(ii) The index of α̂ satisfies

ind (α̂^{t}) = rk (F) − 1

for some t ≥ 1. The action of F on T is geometric. The train track τ has a non-trivial Nielsen path with respect to f^{t}.

It is shown in [BF] that in case (ii) there is precisely one f-orbit of non-trivial indivisible Nielsen paths, and that this path is closed precisely if α̂ is geometric. It has been shown by the second author that (contrary to the geometric case!) in the parageometric case the similarity class of that representative α ∈ Aut (F) of α̂ which is given by the Nielsen path is not an invariant of α̂, but can change when passing over to a different train track representative.

We now turn again to ordinary automorphisms rather than outer ones. For any α ∈ Aut (F) our Theorem 1’ (see (1) above) gives

0 ≤ rk (Fix (α)) ≤ rk (F) and 0 ≤ a(α) ≤ 2rk (F).

We are interested in the case where one of the two summands rk (Fix (α)) or a(α) for the index becomes maximal.

VI.7. Assume rk (Fix (α)) = rk (F). As a consequence we obtain a(α^{t}) = 0 for any t ≥ 1, i.e. α^{t} has no attracting fixed point in δF, by Theorem 1. Hence, by Lemma IV.2, there is no α-invariant tree T as in Theorem II.1 with λ > 1, as it follows from Theorem 2 [GL] that V^{H^{t}}(Q) = V(Q) for some t ≥ 1. Thus we can decompose α as in the cases B2 or B3, since case B1 yields immediately that rk (Fix (α)) < rk (F). We now consider the restriction of α to the vertex group (case B3) or to the vertex groups (case B2) and observe that they also must have fixed subgroup of maximal rank. Hence (by inverting the HNN-composition process described in VI.4) we can proceed inductively to obtain a “normal form” for such automorphisms, see [CT] and [CL 2].

VI.8. Assume that the action on the invariant tree (satisfying λ > 1) is free, and that α is replaced by a sufficiently high power so that H_{α} fixes all orbits of branch points and all orbits of directions at those branch points. Then one obtains ind (α̂) = ind (T). If furthermore T has only one orbit of branch points, then any automorphism α′, with α̂′ = α̂ and with the property that H_{α′} fixes one of the branch points, satisfies

ind (α′) = ind (T).

Thus, if the F -action on T is geometric, one has a(α) = 2rk (F), which is the upper bound. Such examples can be constructed from an irreducible parageometric automorphism through subsequent HNN-compositions chosen so as to connect all similarity classes with positive index (see VI.4). Notice however that the resulting automorphism of a free group of higher rank is of course no longer irreducible. An irreducible automorphism with a(α) = 2rk (F) must be represented by a train track map for α which has one nonclosed Nielsen path connecting the only two fixed points of the map with positive index. Somehow surprisingly such automorphisms exist. An example is given by a ↦ aac^{−1}aac^{−1}b^{−1}, b ↦ bca^{−1}a^{−1}, c ↦ ca^{−1}.

VI.9. Finally, we compare ind (α̂) to ind (α̂^{−1}). It is easy to find examples with ind (α̂) ≠ ind (α̂^{−1}), (and hence ind (α′) ≠ ind (α′^{−1}) for some α′ representing α̂). Such examples occur even among irreducible automorphisms with irreducible powers. In particular there are positive automorphisms α satisfying ind (α) = ind (α̂) = n − \frac{3}{2}, while α̂^{−1} is parageometric and hence satisfies ind (α̂^{−t}) = n − 1 for some t ≥ 1. An example is the automorphism α : a ↦ abc, b ↦ bab, c ↦ cabc, which is the inverse of the one given in VI.8. Notice that this example also satisfies λ(α̂) ≠ λ(α̂^{−1}); for further examples satisfying this last inequality see [BH].

VI.10. Notice that the automorphism α_{3} = α from VI.9 has 11 fixed points on δF, 5 of them being attractive, and 6 repelling. The analogue is true for α_{4} : a ↦ abcd, b ↦ bab, c ↦ cabc, d ↦ dabcd, which has 7 attractive and 8 repelling fixed points at δF. We believe this pattern repeats for all the analogously defined α_{n}.

References

[BF] M. Bestvina, M. Feighn, Outer limits, preprint (1992).

[BH] M. Bestvina, M. Handel, Train tracks for automorphisms of the free group, Ann. Math. 135 (1992), 1–51.

[CDP] M. Coornaert, T. Delzant, A. Papadopoulos, Géométrie et théorie des groupes, Lecture Notes 1441, Springer-Verlag, 1990.

[CL 1] M.M. Cohen, M. Lustig, On the dynamics and the fixed subgroup of a free group auto-

morphism, Inv. Math. 96 (1989), 613–638. [CL 2] M.M. Cohen, M. Lustig, Very small group actions on R-trees and Dehn twist automor-

phisms, Topology 34 (1995), 575–617.

[CL 3] M.M. Cohen, M. Lustig, The conjugacy problem for Dehn twist automorphisms of free

groups, Comm. Math. Helv. (to appear).

[CM] M. Culler, J.W. Morgan, Group actions on R-trees, Proc. Lond. Math. Soc. 55 (1987), 571–604.

[Co] D. Cooper, Automorphisms of free groups have finitely generated fixed point sets, Jour. Alg. 111 (1987), 453–456.

[CT] D.J. Collins, E.C. Turner, All automorphisms of free groups with maximal rank fixed subgroups, Math. Proc. Camb. Phil. Soc. 119 (1996), 615–630.

[DV] W. Dicks, E. Ventura, The group fixed by a family of injective endomorphisms of a free group, Contemp. Math. 195 (1996).

[Ga] F.R. Gantmacher, Théorie des matrices, tome 2, Dunod, 1966.

[GH] E. Ghys, P. de la Harpe (eds.), Sur les groupes hyperboliques d’après Mikhael Gromov, Progress in Mathematics 83, Birkhäuser, 1990.

[GL] D. Gaboriau, G. Levitt, The rank of actions on R-trees, Ann. Sc. ENS 28 (1995), 549– 570.

[GLL] D. Gaboriau, G. Levitt, M. Lustig, A dendrological proof of the Scott conjecture for automorphisms of free groups, Proc. Edinb. Math. Soc. (to appear).

[Le] G. Levitt, La dynamique des pseudogroupes de rotations, Inv. Math. 113 (1993), 633– 670.

[Li] I. Liousse, Feuilletages transversalement affines des surfaces et actions affines de groupes sur les arbres réels, Thèse, Toulouse (1994).

[Lo] J. Los, Pseudo-Anosov maps and invariant train tracks in the disc: a finite algorithm, Proc. London Math. Soc. 66 (1993), 400–430.

[Lu] M. Lustig, Automorphisms, train tracks and non-simplicial R-tree actions, Comm. in Alg. (to appear).

[LuOe] M. Lustig, U. Oertel, Invariant laminations for diffeomorphisms of handlebodies, preprint (1996).

[MaSm] H. Masur, J. Smillie, Quadratic differentials with prescribed singularities and pseudo-Anosov diffeomorphisms, Comm. Math. Helv. 68 (1993), 289–307.

[MoSh] J.W. Morgan, P.B. Shalen, Valuations, trees, and degenerations of hyperbolic structures, I, Ann. Math. 120 (1984), 401–476.

[Pa 1] F. Paulin, Topologie de Gromov équivariante, structures hyperboliques et arbres réels,

Inv. Math. 94 (1988), 53–80. [Pa 2] F. Paulin, The Gromov topology on R-trees, Top. and its Appl. 32 (1989), 197–221. [Pa 3] F. Paulin, Sur les automorphismes extérieurs des groupes hyperboliques, Ann. scient. Éc.

Norm. Sup (to appear).

[Sel] Z. Sela, The Nielsen-Thurston classification and automorphisms of a free group, I, Duke Math. Jour. 84 (1996), 379–397.

[Ser] J.-P. Serre, Arbres, amalgames, SL_{2}, Astérisque 46 (1977), SMF Paris. [Sha 1] P.B. Shalen, Dendrology of groups: an introduction, pp. 265–319 in “Essays in group

theory (S.M. Gersten, ed.)”, MSRI Publ. 8 (1987), Springer Verlag. [Sha 2] P.B. Shalen, Dendrology and its applications, pp. 543–616 in “Group theory from a

geometrical viewpoint (E. Ghys, A. Haefliger, A. Verjovsky, eds.)”, World Scientific, 1991.

[Sho 1] H. Short et al, Notes on word hyperbolic groups, Group theory from a geometrical view-

point (Ghys, Haefliger, Verjovsky, eds.), World Scientific, 1991, pp. 3–63. [Sho 2] H. Short, Quasiconvexity and a theorem of Howson’s, Group theory from a geometrical

viewpoint (Ghys, Haefliger, Verjovsky, eds.), World Scientific, 1991, pp. 168–176.

D.G.: Laboratoire Émile Picard, Umr Cnrs 5580, Université Paul Sabatier, 31062 Toulouse Cedex 4, France.

Current address: Unité de Mathématiques CNRS UMR 128, Ecole Normale Supérieure de Lyon, 69364 Lyon Cedex 07, France

E-mail address: gaboriau@umpa.ens-lyon.fr

A.J, M.L.: Institut für Mathematik, Ruhr-Universität Bochum, Postfach 10 21 48, 4630 Bochum 1, Germany.

E-mail address:

Andre.Jaeger@ruba.rz.ruhr-uni-bochum.de
Martin.Lustig@ruba.rz.ruhr-uni-bochum.de

G.L.: Laboratoire Émile Picard, Umr Cnrs 5580, Université Paul Sabatier, 31062 Toulouse Cedex 4, France.

E-mail address: levitt@picard.ups-tlse.fr