NEWTON’S METHOD OF APPROXIMATION

AN ENDURING MYTH

N. KOLLERSTROM

British Journal for History of Science,1992, 25, 347-54

2004 Note: Newton’s fluxional notation employed a dot over single letters, which for HTML use has here been put in front, as ‘.x,’ e.g. This article looks at how Thomas Simpson’s 1740 method has come to be credited to Isaac Newton, and also at historical exaggerations of the extent to which Newton used the methods of differential calculus.

A resurgence of interest has occurred in ‘Newton’s method of approximation’ for deriving the roots of equations, as its repetitive and mechanical character permits ready computer use (1). If x = α is an approximate root of the equation f(x) = 0, then the method will in most cases give a better approximation as

 α - f(α)/f’(α), (I)

where f’’(x) is the derivative of the function into which a has been substituted (2). Older books sometimes called it ‘the Newton-Raphson method’, although the method was invented essentially in the above form by Thomas Simpson, who published his account of the method in 1740 (3). However, as if through a time-warp, this invention has migrated back in time and is now matter-of-factly placed by historians in Newton’s De analysi of 1669 (4). This paper will describe the steps of this curious historical transposition, and speculate as to its cause.

What is today known as ‘Newton’s method of approximation’ has two vital characteristics : it is iterative, and it employs a differential expression. The latter is simply the derivative f’’(x) of the function, resembling a Newtonian fluxion in being based upon a theory of limits but not conceptually identical with it. The method uses the fundamental equation (I) repetitively, inserting at each stage the (hopefully) more accurate solution. This paper will argue that neither of these characteristics applies to the method of approximate solution developed by Newton in De analysi (5), which also appeared in his De methodis fluxionum et serierum inftnitorum (6), and that the method of approximation published by Joseph Raphson in his Analysis aequationum universalis of 1690 (7) was iterative - indeed was the first such method to be iterative - but was not expressed in derivative or fluxional terms.

NEWTON’S METHOD

D. T. Whiteside described the method of approximate solution given in Newton’s De analysi as ‘essentially an improved version of the procedure, expounded by Viete and simplified by Oughtred’ (8). John Wallis, in the first edition of his Algebra (1685), extolled the method as a fine British achievement (9), and we shall here follow his account of it. Taking the cubic equation y3-2y-5 = 0, as the example given in De analysi, he started with the approximate solution of 2. Let the exact solution be 2+p, where p is small, and substitute 2+p into the equation in place of y. This generates a new cubic equation, namely p3+6p2+10p = 1. As p is small, its powers are ignored, yielding the approximate solution 10p = 1 or p = 0.1. Next, 0.1+q is inserted in place of p to form another new equation, and so on. This is continued to achieve any desired level of accuracy.

That was Newton’s method. To quote from W. Frend’s account of it in 1796, it proceeds by considering the new, or transformed, equation (resulting from the substitution of a+z, or a-z, instead of x, in the original equation)’ (10). It took only the first-order terms in a binomial expansion, a subject with which Newton’s De analysi was much concerned. It did not employ any fluxional calculus.

RAPHSON’S METHOD

When Raphson’s method was announced to the Royal Society in July of 1690, there was emphasis on its innovative nature:Mr Halley related that Mr Ralphson [sic] had Invented a method of Solving all sorts of Aquations [...] and that he had desired of him an Equation of the fifth power to be proposed to him, to which he return’d answers true to Seven Figures in much less time than it could have been effected by the Known methods of Vieta. (11).

Raphson published his method as a tract in 1690. It had a preface referring to Newton, among several other mathematicians, in which Raphson declared that his own method was somewhat similar (‘aliquid simile’) to Newton’s earlier account (12). Raphson removed that preface when publishing his method as a book in 1697. He then referred solely to Viete as the ancestor of his method. Later in the work (section VI) he referred to the English mathematicians Harriot and Oughtred. An Appendix was added, referring amongst other matters to Newton’s binomial theorem.

Raphson presented his method as follows. Taking as an example the equation ba - aaa = c (which we would write as a3 - ba + c = 0), let an approximate solution be g. Then, if a more accurate solution is g+x,

X = (c+ggg-bg) / (b-3gg)

The quotient expression was obtained by a two-step procedure (13). In the above example, one substituted (g + x) for a, then expanded the power terms to give a larger equation; this was a straightforward binomial expansion. The second step was to extract the terms in x: the terms which multiplied x in this example were (b-3gg), and these became the quotient.

Iterating this procedure, Raphson explained, would give any desired level of accuracy. He elaborated his method only within the context of polynomial equations, without attempting to deal with reciprocal or square root functions. His worked examples contained terms up to the seventh power.

This was Raphson’s method, which he said he had derived from Viète. If it sounds odd today, it is because once the calculus technique was established, such ad hoc rules could be forgotten. Nowadays, it is invariably viewed in calculus terms - in the above case, the derivative of (a3 - ba + c) is (3a2 - b) ; one divides the original function by its derivative, substituting the approximate solution g to obtain the increment x, whereby it is improved. For Raphson no such general concept appeared to be available. His book contained many pages of recipes showing how for each specific algebraic expression one could obtain the required quotient : for example the quotient for gggg was 4ggg. However, no general proofs of these recipes were provided; they were obtained using the two-step procedure.

Even after De l’Hopital’s Analyse d’infiniments petits was published in 1696, and rapidly became the textbook on the new Leibnizian differential and integral calculus, Raphson republished his method without alteration. As testimony to the level of British awareness of the new differential or fluxional procedures in the 1690s, this situation could have been referred to by Raphson in his History o f Fluxions published in 1715, though this might well have compromised the staunchly pro-British tenor of its argument.

Newton’s fluxional method, as it featured in Wallis’s 1693 Opera mathematica (14) would hardly have sufficed to deliver the required quotient term : an implicit differentiation method was there outlined which left the time-based fluxions .z and .y embedded in the equation (15). To obtain the gradient of a curve it was necessary further to divide .y by .z, a procedure which only developed in the next century. In 1695, de Moivre was still using the terms ‘fluxion’ and ‘moment’ to represent infinitely small quantities (16). Raphson over this period evidently saw nothing to make him recast his method into a fluxional format for the second edition of his History of Fluxions. In this he was not alone: Halley in 1694 took twelve pages of the Philosophical Transactions giving his method of solving polynomial equations by successive approximation (17), based largely on the method of a Frenchman, Thomas de Lagny (18), and one there finds no sign of the new fluxional method. This should not surprise us, as new ideas take a while to become accepted.

Raphson himself first referred to the fluxional method in his Mathematical Dictionary of 1702 (18). The reference he cited for it was Book II of Wallis’s Opera (1693). His account was polemical, relating to the storm of controversy then gathering: Newton’s fluxional method, Raphson wrote, ‘passes there [Germany] and in France, under the name of Leibniz’s differential calculus’. This reference made no allusion to methods of approximate solution for equations.

The Appendix to the second edition of Raphson’s Analysis (1697) referred to the work of several contemporary mathematicians : Halley, de Lagny, Abraham Sharp (who had found π to fifty places) and then fourthly Isaac Newton. This is the sole reference to Newton in Raphson’s final statement of his method, so let us be clear as to what is there acknowledged. It referred to chapter 91 of Wallis’s Algebra of 1685, where Newton’s procedures for binomial expansion were described (with infinite series for reciprocal functions), and not to chapter 94, which gave Newton’s method of approximation. We may assume that he saw no need to refer to this latter section.

Raphson was there impressed by the new nomenclature for powers of variables that Newton was using, as advocated by Wallis, for example writing aaa as a3. Raphson gave several examples of how his computations could be rewritten in this manner. His Appendix also referred to Newton’s method of infinite series expansions; however, as none of Raphson’s worked examples dealt with fractional or negative powers - which require those expansions - it is doubtful whether he can be said to have incorporated these into his method. It is quite evident that Raphson in his second edition of 1697 did not make any acknowledgement to Newton of the kind that subsequent historians have either alleged that he did, or assumed that he should have done.

SIMPSON’S METHOD

Thomas Simpson, FRS (1710-61), was a well-known British interpolationist, author of ‘ Simpson’s rule’ for obtaining the area under a curve and other results. Writing in 1740 he described ‘A new Method for the Solution of Equations’, making no reference to any predecessors, and affirming that: ‘as it is more general than any hitherto given, it cannot but be of considerable use’ (20). It was indeed. His fine opening words were ‘Take the Fluxion of the given Equation...’, from which he proceeded to a version of the rule as presented in (1), using fluxions. His instructions here were: ‘... and having divided the whole by .x, let the Quotient be represented by A’. Fluxions were taken in the manner that Newton described to Wallis in 1692, which left .x and .y terms on each side of the equation. Dividing though by .x left what would nowadays be called the derivative of the function on the right-hand side, and dy/dx on the left. This differential expression was what Simpson referred to as ‘A’. In this manner, he applied fluxions to the approximation method.

Simpson cited five examples, including a cubic equation, a square root function, a reciprocal and an exponential function (21). It is evident that Simpson had a general command of the fluxional technique, whereby he could obtain the quotient term for the approximation formula.

Some might object, Simpson commented, that the method of fluxions ‘being a more exalted Branch of the Mathematics, cannot be so properly applied to what belongs to common algebra’ (22). This indicates that he believed he was being innovative in applying the method of fluxions to this area of mathematics. His use of a fluxion in this manner sufficiently resembles the modern formulation for him to be credited (I suggest) as inventor of the method.

THE METHODS COMPARED

In the eighteenth century there was debate over whether the Newton or the Raphson method was preferable. The ‘Observations on Mr Raphson’s method’ (1796) by W. Frend compared their relative merits, and concluded that:

with respect to the simplicity and conception of the two methods, Mr Raphson’s method seems to be preferable to Sir Isaac Newton’s; because the former always refers back to the original equation x3-2x = 5, whereas the latter method refers to the preceding transformed equation 10z+6x3 = 1, which has more terms and larger coefficients than the original equation ...I consider Mr Raphson’s method of resolving them [equations] as, upon the whole, more convenient than that of Sir Isaac Newton (23).

This view was echoed in virtually identical terms by Francis Maseres, a Fellow of the Royal Society, in a tract of nine pages comparing the two methods (24). J. L. Lagrange’s influential treatise, Resolution des equations numeriques of 1798, discussed the two methods. It referred to Newton’s method of approximation as being well known, and refined and generalized the Newtonian method of De analysi, though without reference to fluxions or differentials. Lagrange expressed surprise that Raphson had not referred to Newton’s earlier work, taking the view that ‘ces deux methodes ne sont au fond que le meme presentee differemment’, though conceding that Raphson’s method was ‘plus simple que celle de Newton’, because ‘on pent se dispenser de faire continuellement de nouvelles transformees’ (25).

These remarks we find in one of the notes at the end of Lagrange’s treatise. Its main text was composed in 1767-68, and twelve notes were added in the 1790s. These notes used the ‘ f’(x)’ notation for the ‘fonction dérivée’. He had introduced it as part of his algebraic foundation of the calculus, based upon expanding a function f(x+h) in powers of h and defining these ‘fonctions’ from the coefficients of the powers. This was his own version of the differential calculus to replace both Newton’s and Leibniz’ (28); yet it is striking that he did not use it in his note 5 comparing the approximation methods - even though he did apply it freely in various other of these notes (27). As Raphson had done a century earlier, Lagrange treated the approximation methods solely in algebraic terms. This may remind us how very innovative Simpson was being, in applying the fluxional technique within this algebraic context.

THE MYTH BLOSSOMS

In the early nineteenth century, the mathematician Joseph Fourier presented the method in terms of the now-universal f’(x) calculus notation, describing it as ‘le méthode newtonienne’ (28). Fourier’s writings on equations became very well known. The British mathematicians Burnside and Panton referred to the method, using the language of calculus, as being that of Newton and Lagrange, without mentioning Raphson. They did refer to Simpson and the Bernoullis as having ‘occupied themselves’ with the problem (29). Similarly in Germany, Runge gave the method in Leibnizian form, attributing it to Newton (30). Moritz Cantor reviewed the approximation methods of Newton, Raphson, Halley and de Lagny, describing Raphson as ‘an absolute admirer and imitator of Newton’, whose approximation method ‘greatly resembled that of Newton’ (31).

Reviewing the situation in 1911, Florian Cajori concluded that the method ought properly to be called the ‘Newton-Raphson method’ (32); however, no person in the seventeenth or eighteenth centuries adopted such a view. Cajori’s grounds for referring to the method as ‘the Newton-Raphson method’ may have been his view that ‘If r is the approximation already reached, then Newton uses a divisor which in our modern notation takes the form f’’(r)’.(33). However, the Newtonian method does not inherently employ a divisor, let alone one equivalent to f’’(r).

A recent appreciation of Joseph Raphson discussed the historically perceived difference between the two methods, concluding: ‘it is actually Raphson’s simpler (and therefore superior) method, not Newton’s, that lurks inside millions of modern computer programs’ (34). In support of this argument it presented the familiar claim that Raphson’s method involved ‘calculation of the first derivative’, quoting the differential-based equation given at the start of this article. The historical record hardly supports such a viewpoint. The method of approximation inside computer programs is surely that of Simpson.

Such attitudes endure to this day, to be found even in histories of mathematics. Boyer’s History of Mathematics (1968) affirmed that ‘Newton’s Method’ for the approximate solution of equations could be found in De analysi (35), citing its modern formulation in terms of a derivative f’(x). A marginally more accurate version has appeared in Makers of Mathematics by Stuart Hollingdale (1989), which correctly described the method of approximation given in De analysi, but then blithely asserted, ‘Newton also devised an iterative method ... first published in its original form by Joseph Raphson in 1690’ (36).

THE LURE OF MYTH

The seeds of lasting confusion were sown by Wallis in his Opera of 1693 (37): he received in August 1692 historic letters from Newton, now lost (38), containing the recipe for what would nowadays be called implicit differentiation in fluxional terms of an equation, using the newly-invented dot notation (39); he published that method without acknowledging a contemporary source, and alleged that the method was present in Newton’s letters of the 1670s sent to Leibniz, which was scarcely the case (40). That act needs to be seen within the context of the controversy then beginning over the genesis of the new calculus methods. To quote Whiteside, ‘The letters to Wallis in 1692... [were] the first significant announcement to the world at large of the power of Newton’s fluxional method’ (41).

Modern scholarship has located the fairly limited extent to which Newton did compose differential equations, in the early 1690s (42). These were reformulations of dynamical issues from his Principia, and did not include methods of approximate solution of equations. It seems that only at the tercentenary of these events can myth and fact be disentangled. Disputes over the birth of calculus have led mathematicians to locate such achievements at a too-early period. The myth we have surveyed is a legacy from that dispute.

Taking the time-honoured view that Raphson used differentials or fluxions in his method, where was he supposed to have got them from? This always remained unspecified. Prior to Wallis’s 1693 publication, it is not evident that there was a published source from which British mathematicians could have derived such a method, had they so wished.

It was, we have here argued, unequivocally the method of approximation invented by Thomas Simpson that Fourier restated using a derivative notation, and which has somehow come to gravitate within a Newtonian orbit. I found no source which credited Simpson as being an inventor of the method. None the less, one is driven to conclude that neither Raphson, Halley nor anyone else prior to Simpson applied fluxions to an iterative approximation technique.

References

1. H. Peitgen and P. Richter, The Beauty of Fractals, Berlin, 1986, 18.

2. See e.g. C. Tranter and C. Lambe, Advanced Level Mathematics, 4th edn, London, 1980, 302.

3. T. Simpson, Essays ... on Mathematics, London, 1740, 81.

4. J. Pepper, ‘Newton’s mathematical work’, in Let Newton Be! (ed. J. Fauvel et al.), Oxford, 1988, 63-80: ‘Newton made a major breakthrough [in De analysi] by introducing what is now known as the Newton-Raphson method’ (p. 73). H. Goldstine, A History of Numerical Analysis from the Sixteenth Century through theNineteenth, Springer-Verlag, New York, 1977,64-7. D. M. Burton, The History o f Mathematics, an Introduction, 1986, 408.

5. I. Newton, De analysi (ed. W. Jones), London, 1711; The Mathematical Papers of Isaac Newton (ed. D. T.Whiteside), Cambridge, 1968, ii, 206-47. see 218-19.

6. I. Newton, De methodis fluxionum et serierum in finitorum’, London, 1736 (English translation J. Colson), Whiteside, op. cit. (5), iii, 32-353; see 43-7, ‘The reduction of affected equations’.

7. J. Raphson, Analysis aequationum universalis..., London, 1690.

8. Whiteside, op. cit. (5), ii, 218.

9. J. Wallis, A Treatise of Algebra both Historical and Practical, London, 1685, 338.

10. W. Frend, The Principles of Algebra, London, 1796, 456.

11. Journal Book of the Royal Society of London, 30 July 1690.

12. Raphson, op. cit. (7), Preface. Goldstine said of this 1690 work, ‘Here Raphson acknowledges Newton as the source of the procedure’ (Goldstine, op. cit. (4), 64). That is not the view here taken.

13. Raphson, op. cit. (7), 1-2.

14. J. Wallis, Opera Mathematica, ii, London, 1693, 391-6.

15. For a different view see H. Bos, ‘Newton, Leibniz and the Leibnizian tradition’ in From Calculus to Set Theory 1630-1910 (ed. I. Grattan-Guinness), London, 1980, Ch. 2, 49-93, on 88.

16. A. de Moivre, ‘Doctrinae fluxionum...’, Philosophical Transactions (1695), 19, 52-7. See F. Cajori, Conceptions of Limits and Fluxions in Great Britain from Newton to Woodhouse, 1919, Chicago and London, 39.

17. E. Halley, ‘Methodus nova accurata et facilis inveniendi radices aequationum...’, Philosophical Transactions (1694), 18, 136-48.

18. T. F. de Lagny, Methodes nouvelles et abregeds pour !’extraction et !’approximation des racines, Paris, 1734.

19. J. Raphson, A Mathematical Dictionary, London, 1702. 20 T. Simpson, Essays... on Mathematics, 1740, Preface, vii.

21 Ibid., 83-6.

22 Ibid., vii.

23 W. Frend, op. cit. (10), 456, 492.

24 F. Maseres, ‘On Mr Raphson’s Method of Resolving Affected Equations by Approximation’, in Bernoulli’s Mathematical Tracts, 1795, published by F. Maseres, London, 577-86, on 585.

25 J. L. Lagrange, Note V, ‘Sur la methode d’approximation donnee par Newton’, in Traite de la resolution des equations numeriques, 1st edn, Paris, 1798; 2nd edn, 1808, reprinted 1826, 122.

26 Grattan-Guinness, op. cit. (15), Ch. 3, I. Grattan-Guinness, ‘The emergence of mathematical analysis and its foundational progress, 1780-1880’, p. 115.

27 Lagrange, op. cit. (25), 130-52.

28 J. B. J. Fourier, Analyse des equations determinees, Paris, 1831, 169, 173 and 177.

29 W. S. Burnside and A. W. Panton, The Theory of Equations, London, 1881, Note B, 384-6.

30 C. Runge, ‘Separation and Approximation der Wurzeln’, Encyk. der Math. Wissenschaften, 1900, 1,

404-48, article IB3a (pp. 433-5).

31 M. Cantor, Geschichte der Mathematik, Leipzig, 1898, iii, 114-15; also 2nd edn (1901), 119-20.

32 F. Cajori, ‘Historical note on the Newton-Raphson method of approximation’, American Mathematical Monthly (1911), 18, 29-32, on 30.

33 Ibid., 31.

34 D. J. Thomas, ‘Joseph Raphson, F.R.S.’, Notes Rec. Roy. Soc. London (1990), 44, 151-67, on 155. Thomas here mistakenly claims (p. 155) that Newton ‘never published his version’ [of approximation method], but see note 6 above. In addition the text of De analysi was reprinted in the Commercium epistolicum of 1713.

35 C. Boyer, A History of Mathematics, Princeton, 1968, reprinted 1980, 449.

36 S. Hollingdale, Makers of Mathematics, London, 1989, 179.

37 J. Wallis, op. cit. (14), 390.

38 The Correspondence of Isaac Newton, London, 1961, iii, 222-8.

39 D. T. Whiteside, ‘The mathematical principles underlying Newton’s Principia’, Journal for the History of Astronomy (1790), 1, 116-38, on 119.

40 A. R. Hall, Philosophers at War, Cambridge, 1980, 94-6.

41 D. T. Whiteside, ‘Essay review of The Correspondence of Isaac Newton, Vol. III’, History of Science (1962), 1, 97.

42 Whiteside, op. cit. (39), 119.  