Mathematics | Rod Carvalho

Archive for the ‘Mathematics’ Category

Interview with László Lovász

May 11, 2013

On November 28, 2011, Avi Wigderson interviewed László Lovász. A total of 27 short videos can be watched at the Simons Foundation website. I particularly liked the discussion on the LLL algorithm, and also on semidefinite relaxations.

__________

The social construction of Hungarian genius

Tags:Academic Politics, Alfred Tarski, Approximation Algorithms, Avi Wigderson, Biographies, Channel Capacity, Combinatorics, Extremal Graph Theory, Graph Limit Theory, Graph Theory, History of Mathematics, Hungarian Mathematicians, Hungarian Mathematics, International Mathematical Union, Interviews, László Lovász, Lenstra–Lenstra–Lovász Algorithm, Mathematicians, Mathematics Competitions, Mathematics Education, Open Problems, P vs NP, Parenting, Paul Erdős, Regularity Lemma, SDP Relaxations, Simons Foundation, Sociology of Mathematics, Videos
Posted in Combinatorics, Computer Science, Graph Theory, Mathematics | Leave a Comment »

Composing an endofunction with itself

November 30, 2012

Suppose we are given an endofunction $f : \mathcal{X} \to \mathcal{X}$ . Let us introduce a new function $f^{(n)}$ , where $n \geq 0$ , which we define recursively as follows

$f^{(k)} = f \circ f^{(k-1)}, \qquad{} f^{(0)} = \text{id}$

where $\text{id} : \mathcal{X} \to \mathcal{X}$ is the identity function on $\mathcal{X}$ , i.e., $\text{id} (x) = x$ for all $x \in \mathcal{X}$ . In Haskell, it is easy to create a higher-order function compose that composes an endofunction $f$ with itself $n$ times:

compose :: (a -> a) -> Integer -> (a -> a)
compose f 0 = (\x -> x)
compose f k = f . (compose f (k-1))

where the identity function is defined anonymously. If you happen to dislike anonymity in your functions, here is another implementation:

compose :: (a -> a) -> Integer -> (a -> a)
compose f 0 = identity
compose f k = f . (compose f (k-1))

identity :: a -> a
identity x = x

There are many other alternatives. Take a look at this Stack Overflow thread. I particularly liked Reid Barton‘s elegant solution using a right-fold:

compose f n = foldr (.) id (replicate n f)

where id = (\x -> x). Beautiful, isn’t it? If this one-liner does not convert you to Haskell, then nothing will.

__________

Example

Let $\mathbb{Z}_{64} := \{0, 1,\dots,63\}$ , and let function $f$ be defined by

$\begin{array}{rl} f : \mathbb{Z}_{64} &\to \mathbb{Z}_{64}\\ n &\mapsto n+1 \pmod{64}\end{array}$

This function corresponds to a cyclic shift. Therefore, we expect to have $f^{(64)} = \text{id}$ . Let $g := f^{(32)}$ and $h := f^{(64)}$ . We load the script above into GHCi, and voilà, we have the session:

*Main> let ns = [0..63]
*Main> let f =  (\n -> (n+1) `mod` 64)
*Main> map f ns
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,
 17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,
 33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,
 49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,0]

So far, so good! Let us take a look at $g$ and $h$ :

*Main> let g = compose f 32
*Main> let h = compose f 64
*Main> map g ns
[32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,
 48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,
 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,
 16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]
*Main> map h ns
[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,
 16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,
 32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,
 48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]

Indeed, as we expected, we have $f^{(64)} = \text{id}$ .

__________

Everything in this post is elementary. But now that the elements are solid, we can dare to attempt more sophisticated things. Like what? Like composing lists of (distinct) functions. I am thinking of permutations. I am thinking of perfect shuffles.

Tags:Function Composition, Haskell, Higher-Order Functions
Posted in Haskell, Mathematics | 1 Comment »

Generalizing Heron’s method

November 25, 2012

Suppose we are given a positive rational number $y$ . We would like to compute an approximation of the $n$ -th root of $y$ , which we denote by $\sqrt[n]{y}$ . If $n = 2$ , we can use Heron’s method to approximate $\sqrt{y}$ , as we saw a few days ago. But, what if we have that $n > 2$ ? Is it possible to generalize Heron’s method?

Indeed, it is. Let us introduce a function $f_{n,y} : \mathbb{R} \to \mathbb{R}$ , defined by $f_{n,y} (x) = x^n - y$ , where $n \geq 2$ and $y > 0$ . By construction, the positive zero of function $f_{n,y}$ is $\sqrt[n]{y}$ . Do note that if $n$ is odd, then $-\sqrt[n]{y}$ is not a zero of $f_{n,y}$ . Recall that Newton’s method uses the recurrence relation $x_{k+1} = g_{n,y} ( x_k )$ , where function $g_{n,y} : \mathbb{R} \to \mathbb{R}$ is defined by

$g_{n,y} (x) = x - \displaystyle\frac{f_{n,y} (x)}{f_{n,y}' (x)}$

where $f_{n,y}'$ is the first derivative of $f_{n,y}$ . Hence, we obtain

$g_{n,y} (x) = x - \displaystyle\frac{f_{n,y} (x)}{f_{n,y}' (x)} = x - \displaystyle\left(\frac{x^n - y}{n x^{n-1}}\right) = \displaystyle\frac{1}{n} \left( (n-1) x + \frac{y}{x^{n-1}}\right)$

Thus, we have the generalized Heron’s method

$x_{k+1} = \displaystyle\frac{1}{n} \left( (n-1) x_k + \frac{y}{x_k^{n-1}}\right)$

If we make $n = 2$ , we obtain the standard Heron’s method, as we expected. The following Haskell script implements this generalized Heron’s method using arbitrary-precision rational numbers (included in the Data.Ratio library) to represent $y$ and the sequence of approximations $(x_k)_{k \in \mathbb{N}_0}$ :

import Data.Ratio

-- define generalized Heron map
g :: Integer -> Rational -> Rational -> Rational
g n y x = (1 % n) * ((fromIntegral (n-1) * x) + (y / (x^(n-1))))

-- initial approximation
x0 :: Rational
x0 = 1 % 1

-- list of approximations of the n-th root of y
roots :: Integer -> Rational -> [Rational]
roots n y | n  > 1 = iterate (g n y) x0
          | n <= 1 = error "Invalid value of n!!"

We load this script into GHCi. Let us now perform some numerical experiments.

__________

Experiment #1

We would like to compute a rational approximation of $\sqrt{2}$ . We have $n = 2$ and $y = 2$ . Here is a very brief GHCi session:

*Main Data.Ratio> take 6 (roots 2 2)
[1 % 1,3 % 2,17 % 12,577 % 408,665857 % 470832,
 886731088897 % 627013566048]

We obtain the following rational approximation after five iterations

$x_5 = \displaystyle\frac{886731088897}{627013566048} \approx \sqrt{2}$

which is the same approximation I obtained twice before.

__________

Experiment #2

We would like to compute a rational approximation of $\sqrt{5}$ . We have $n = 2$ and $y = 5$ . Here is a very brief GHCi session:

*Main Data.Ratio> take 7 (roots 2 5)
[1 % 1,3 % 1,7 % 3,47 % 21,2207 % 987,4870847 % 2178309,
 23725150497407 % 10610209857723]

We obtain the following rational approximation after six iterations

$x_6 = \displaystyle\frac{23725150497407}{10610209857723} \approx \sqrt{5}$

How good is this approximation? Let us check:

*Main Data.Ratio> (23725150497407 % 10610209857723)^2 - 5
4 % 112576553224922323902744729
*Main Data.Ratio> 4 / 112576553224922323902744729
3.5531377408652764e-26

Not bad! We could now obtain a rational approximation of the famous golden ratio $\varphi := \frac{1 + \sqrt{5}}{2}$ , as follows:

*Main Data.Ratio> let xs = roots 2 5
*Main Data.Ratio> let phis = map (/2) (map (+1) xs)
*Main Data.Ratio> take 7 phis
[1 % 1,2 % 1,5 % 3,34 % 21,1597 % 987,3524578 % 2178309,
 17167680177565 % 10610209857723]

We obtain the following rational approximation after six iterations

$\tilde{\varphi}_6 = \displaystyle\frac{17167680177565}{10610209857723} \approx \varphi$

How good is this approximation? Let us recall that the golden ratio is one of the solutions of the equation $x^2 - x - 1 = 0$ . Hence, we can check how close to zero $\tilde{\varphi}_6^2 - \tilde{\varphi}_6 - 1$ is:

*Main Data.Ratio> let phi6 = 17167680177565 % 10610209857723
*Main Data.Ratio> phi6^2 - phi6 - 1
1 % 112576553224922323902744729
*Main Data.Ratio> 1 / 112576553224922323902744729
8.882844352163191e-27

I would say that is fairly close.

__________

Experiment #3

We now would like to compute a rational approximation of $\sqrt[3]{10}$ . We have $n = 3$ and $y = 10$ . Here is a very brief GHCi session:

*Main Data.Ratio> take 6 (roots 3 10)
[1 % 1,4 % 1,23 % 8,4909 % 2116,55223315303 % 25495981298,
83759169926117983945469262167029 % 38876457805393768546966848104041]

We obtain the following rational approximation after five iterations

$x_5 = \displaystyle\frac{83759169926117983945469262167029}{38876457805393768546966848104041} \approx \sqrt[3]{10}$

Is this approximation any good? Let us check:

*Main Data.Ratio> let x5 = (roots 3 10) !! 5 
*Main Data.Ratio> x5^3
5876207108075025254603649658428809744645894398375582704
87986732898174481543266076676033790365389 % 58757060813
2677736272189042625165443146909261971794220978628909441
43919908404893015241356540921
*Main Data.Ratio> 5876207108075025254603649658428809744
6458943983755827048798673289817448154326607667603379036
5389 / 587570608132677736272189042625165443146909261971
79422097862890944143919908404893015241356540921
10.000852709004354

This is somewhat disappointing. The rational approximation $x_5$ is a ratio of two enormously long integers, but we do not have such a small error. Note that $x_5^3 - 10 \approx 10^{-3}$ . We can do some quick error analysis. Let $f_{3,10} (x) = x^3 - 10$ . The 1st order Taylor approximation of $f_{3,10}$ around $x_5$ evaluated at $x = \sqrt[3]{10}$ is

$f_{3,10} (\sqrt[3]{10}) \approx f_{3,10} (x_5) + f_{3,10}' (x_5) (\sqrt[3]{10} - x_5)$

and, since $f_{3,10} (\sqrt[3]{10}) = 0$ , we obtain the estimate of the error

$x_5 - \sqrt[3]{10} \approx \displaystyle\frac{f_{3,10} (x_5)}{f_{3,10}' (x_5)}$

In other words, the output deviation $f_{3,10} (x_5) - f_{3,10} (\sqrt[3]{10})$ should be divided by the slope $f_{3,10}' (x_5)$ to obtain an estimate of the magnitude of the approximation error $x_5 - \sqrt[3]{10}$ . Let us use GHCi yet once again:

*Main Data.Ratio> let error = (x5^3 - 10) / (3 * x5^2)
*Main Data.Ratio> error
1670089160826306272530773923851043922672595525468316978
5941152245094153072382174540074985393 % 272741620880842
8590518540423731512512814773109630109912907276686930689
12248623218095571481624481
*Main Data.Ratio> 1670089160826306272530773923851043922
6725955254683169785941152245094153072382174540074985393
 / 2727416208808428590518540423731512512814773109630109
91290727668693068912248623218095571481624481
6.123338108179484e-5

This is still quite a huge error, I would say.

__________

Bonus Experiment

Let us go crazy and attempt to compute a rational approximation of $\sqrt[100]{100}$ . We have $n = 100$ and $y = 100$ . We expect the rational numbers to quickly become ratios of astronomically long integers. Therefore, we will not be showing any rational approximations on GHCi. Here is something scary:

*Main Data.Ratio> let x3 = (roots 100 100) !! 3
*Main Data.Ratio> let error = (x3^100 - 100) / (100 * x3^99)
*Main Data.Ratio> let error_float = fromIntegral (numerator error) 
                                  / fromIntegral (denominator error)
*Main Data.Ratio> error_float

After some 20 minutes without an output, I decided to abort this experiment. If you want to show x3, you can, but you will see thousands of digits. Thousands! You have been warned.

Tags:Arbitrary-Precision Arithmetic, Data.Ratio, Golden Ratio, Haskell, Heron’s Method, Newton’s Method, Numerical Methods, Rational Approximations, Rational Arithmetic
Posted in Haskell, Numerical Methods | 4 Comments »

Heron’s method using integer arithmetic

November 22, 2012

Suppose we are given a positive rational number $y$ , whose square root $\sqrt{y}$ we would like to compute. To be more precise, we would like to compute an approximation of $\sqrt{y}$ .

Let $x_k \neq 0$ be an approximation of $\sqrt{y}$ . Hence, we have that $\sqrt{y}$ is the geometric mean of $x_k$ and $y / x_k$ . If we replace the geometric mean of these two numbers by their arithmetic mean, we hopefully obtain a “better” approximation $x_{k+1}$ , as follows

$x_{k+1} = \displaystyle\frac{1}{2} \left( x_k + \frac{y}{x_k}\right)$

If all goes well, then $x_N$ will be “sufficiently close” to $\sqrt{y}$ for a “sufficiently large” integer $N$ . This algorithm is the famous Heron’s method, devised by Heron of Alexandria some 2000 years ago. Some call it the Babylonian method, which suggests that the algorithm may be some 4000 years old. It’s not merely vintage, it’s antique.

Nonetheless, this is not a post on archaeology. We don’t want to sit around and talk about Heron’s method, we want to actually implement it. What programming language should we use? Let us use Haskell. How are we going to represent $y$ and the sequence of approximations $(x_k)_{k \in \mathbb{N}_0}$ ?

__________

Heron’s method using floating-point arithmetic

Though millions of people have already implemented Heron’s method using floating-point arithmetic (I implemented it in C in late 2000), one more implementation would do no harm:

y :: Floating a => a
y = 2

-- define Heron map
g :: Floating a => a -> a
g x = 0.5 * (x + y/x)

-- initial approximation
x0:: Floating a => a
x0 = 1

-- list of approximations
xs :: Floating a => [a]
xs = iterate g x0

Let us load this script into GHCi. Here’s a GHCi session:

*Main> take 6 xs
[1.0,1.5,1.4166666666666665,1.4142156862745097,
 1.4142135623746899,1.414213562373095]
*Main> let x5 = xs !! 5
*Main> x5**2 - 2
-4.440892098500626e-16

We thus have the following approximation after five iterations

$x_5 = 1.414213562373095 \approx \sqrt{2}$

__________

Heron’s method using rational arithmetic

Let us now use rational numbers of arbitrary precision. The following script uses the Data.Ratio library:

import Data.Ratio

y :: Rational
y = 2

-- define Heron map
g :: Rational -> Rational
g x = 0.5 * (x + y/x)

-- initial approximation
x0:: Rational
x0 = 1 % 1

-- list of approximations
xs :: [Rational]
xs = iterate g x0

Let us load this script into GHCi. Here’s a GHCi session:

*Main> take 6 xs
[1 % 1,3 % 2,17 % 12,577 % 408,665857 % 470832,
 886731088897 % 627013566048]
*Main> let x5 = xs !! 5
*Main> x5*x5 - 2
1 % 393146012008229658338304
*Main> 1 / 393146012008229658338304
2.5435842395854372e-24

Note that the error is eight orders of magnitude smaller than in the previous case (where we used finite-precision floating-point numbers). We pay for more accuracy with more computation. Thus, after five iterations, we have the following rational approximation

$x_5 = \displaystyle\frac{886731088897}{627013566048} \approx \sqrt{2}$

This is probably not news to you, my dear reader. Last April, I did, in fact, implement Heron’s method under the disguise of Newton’s method (see the appendix).

__________

Heron’s method using integer arithmetic

A rational number $y \in \mathbb{Q}$ can be expressed as a fraction $a / b$ , where $a$ and $b$ are integers and $b \neq 0$ . Note that $a / b$ can (obviously?) be represented as a pair of integers $(a,b)$ . Since Haskell has arbitrary-precision integers, we can implement Heron’s method using pairs of arbitrary-precision integers to represent $y$ and the sequence of approximations $(x_k)_{k \in \mathbb{N}_0}$ .

Let $y := a / b$ and $x_k := p_k / q_k$ . The recurrence relation

$x_{k+1} = \displaystyle\frac{1}{2} \left( x_k + \frac{y}{x_k}\right)$

can thus be rewritten in the following form

$\displaystyle\frac{p_{k+1}}{q_{k+1}} = \frac{1}{2} \left( \frac{p_k}{q_k} + \frac{a}{b} \frac{q_k}{p_k} \right) = \frac{b \, p_k^2 + a \, q_k^2}{2 b \, p_k \, q_k}$

We then have two coupled recurrence relations

$\left[\begin{array}{c} p_{k+1}\\ q_{k+1}\end{array}\right] = \left[\begin{array}{c} b \, p_k^2 + a \, q_k^2\\ 2 b \, p_k \, q_k\end{array}\right]$

where $a$ , $b$ , $p_k$ and $q_k$ are integers. Here is a Haskell script that implements this vector recurrence relation:

a, b :: Integer
a = 2
b = 1

-- define Heron map
g :: (Integer,Integer) -> (Integer,Integer)
g (p,q) = (b * p^2 + a * q^2, 2 * b * p * q)

-- initial approximation
p0, q0 :: Integer
p0 = 1
q0 = 1

-- list of approximations
xs :: [(Integer,Integer)]
xs = iterate g (p0,q0)

Let us load this script into GHCi. Here’s a GHCi session:

*Main> take 6 xs
[(1,1),(3,2),(17,12),(577,408),(665857,470832),
 (886731088897,627013566048)]
*Main> let x5 = xs !! 5
*Main> x5
(886731088897,627013566048)

Hence, after five iterations, we obtain the very same rational approximation we obtained in the previous case (where we used rational numbers of arbitrary-precision)

$x_5 = \displaystyle\frac{886731088897}{627013566048} \approx \sqrt{2}$

It should be noted, however, that this implementation using arbitrary-precision integers is different in one very important aspect from the one using arbitrary-precision rational numbers. What is the difference? Let GHCi answer this question:

*Main> import Data.Ratio
*Main Data.Ratio> 2 % 2
1 % 1
*Main Data.Ratio> 4 % 4
1 % 1
*Main Data.Ratio> 3 % 6
1 % 2

The main difference between the two implementations is that when we use arbitrary-precision rational numbers, there is automatic reduction of every fraction to its irreducible form, which requires computing the greatest common divisor (GCD) of the numerator and denominator of each rational number at each iteration. This has a computational cost, of course, but it also has the benefit of avoiding reducible fractions (which are inefficient representations of rational numbers). When we use pairs of arbitrary-precision integers, we do not compute the GCD (although we could, if we wanted to), but we may have to work with pairs of unnecessarily long integers (that correspond to reducible fractions), which is potentially dangerous.

For example, suppose that we replace the first three lines in the script above with the following three lines:

a, b :: Integer
a = 2000
b = 1000

We run the script again. Here’s yet another GHCi session:

 
*Main> take 6 xs
[(1,1),(3000,2000),(17000000000,12000000000),
(577000000000000000000000,408000000000000000000000),
(665857000000000000000000000000000000000000000000000,
470832000000000000000000000000000000000000000000000),
(8867310888970000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000,
62701356604800000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000)]
*Main> let x5 = xs !! 5
*Main> x5
(8867310888970000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000,
62701356604800000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000)

Frankly, I am shocked! Multiplying $a$ and $b$ by $1000$ results in $p_5$ and $q_5$ being multiplied by $10^{90}$ . I may have made a mistake counting all those zeros, but it’s a lot more zeros than I expected.

__________

Appendix: from Heron to Newton

Interestingly, Heron’s method can be derived from the no less famous Newton’s method. Let us introduce a function $f : \mathbb{R} \to \mathbb{R}$ , defined by $f (x) = x^2 - y$ . By construction, the zeros of function $f$ are $\sqrt{y}$ and $-\sqrt{y}$ . Recall that Newton’s method uses the recurrence relation $x_{k+1} = g ( x_k )$ , where function $g : \mathbb{R} \to \mathbb{R}$ is defined by

$g (x) = x - \displaystyle\frac{f (x)}{f' (x)}$

where $f'$ is the first derivative of $f$ . Hence, we obtain the following

$g (x) = x - \displaystyle\frac{f (x)}{f' (x)} = x - \displaystyle\left(\frac{x^2 - y}{2 x}\right) = \displaystyle\frac{1}{2} \left( x + \frac{y}{x}\right)$

which is the recurrence relation used in Heron’s method.

Tags:Arbitrary-Precision Arithmetic, Data.Ratio, Haskell, Heron’s Method, Integer Arithmetic, Newton’s Method, Numerical Methods, Rational Approximations, Rational Arithmetic
Posted in Haskell, Numerical Methods | 3 Comments »

Combinatorial Explosion

September 16, 2012

An amusing Japanese cartoon on combinatorial explosion:

Tastefully done, I would say. The target audience consists not of this blog’s readers, but rather of their children.

Hat tip: Michael Lugo

Tags:Combinatorial Explosion, Combinatorics, Computational Complexity, Kindergarten Mathematics, Questions kids ask
Posted in Art, Combinatorics, Computer Science, Fun | 2 Comments »

Rod Carvalho

Archive for the ‘Mathematics’ Category

Interview with László Lovász

Composing an endofunction with itself

Generalizing Heron’s method

Heron’s method using integer arithmetic

Combinatorial Explosion

Pages

Categories

Top Posts

Recent Posts

Recent Comments

Tag Cloud

Archives

Blog Stats

Artists

Blogs

Haskell

Math Blogs

MATLAB

Python

Robotics

Software

Symbolic Computation

Powered by FeedBurner

Meta

Follow “Rod Carvalho”

May 2013
S	M	T	W	T	F	S
« Dec
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

Rod Carvalho

Archive for the ‘Mathematics’ Category

Interview with László Lovász

Composing an endofunction with itself

Generalizing Heron’s method

Heron’s method using integer arithmetic

Combinatorial Explosion

Pages

Categories

Top Posts

Recent Posts

Recent Comments

Tag Cloud

Archives

Blog Stats

Artists

Blogs

Haskell

Math Blogs

MATLAB

Python

Robotics

Software

Symbolic Computation

Category Cloud

Powered by FeedBurner

Meta

Follow “Rod Carvalho”