It cannot be a complete coincidence that several outstanding logicians of the 20th century found shelter in asylums at some time in their lives.

-Gian-Carlo Rota (1932-1999)

Logicomix: an Epic Search for Truth is an ambitious and highly original graphic novel on the heroic quest to make the foundations of Mathematics exact, consistent, and complete. The narrator and protagonist of the story is the great logician and philosopher Bertrand Russell. Some of the other characters are David Hilbert, Henri Poincaré, Georg Cantor, Gottlob Frege, Alfred North Whitehead, Kurt Gödel, Ludwig Wittgenstein, and John von Neumann. It is a book about ideas, passions, and madness.

[ image courtesy of Logicomix ]

The novel was written by Apostolos Doxiadis and Christos Papadimitriou. Alecos Papadatos did the drawings, and Annie Di Donna added color to them. Here are some videos on the making of Logicomix:

[ part 1 | part 2 | part 3 ]

If you happen to be interested in this topic, I would suggest you read Greg Chaitin’s nice essay on Computers, Paradoxes and the Foundations of Mathematics [pdf].

__________

Reviews:

• Bertrand Russell’s mathematical quest adds up to unlikely graphic novel hit

• Bertrand Russell: the thinking person’s superhero

• The sorrows of young Bertie, and the great quest

• Mathematics has never been so exciting

• Algorithm and Blues

• Truth for beginners
  

When I was an undergraduate student I took a course on Machining. I did not enjoy to study the details of how various machine tools are built or how they work, but I had a lot of fun creating machining sequences, which required ingenuity. The regular visits to the machine shop were also fun. After all, it’s not everyday that one gets to see an object taking shape right before one’s eyes :-)

The instructor would give us a certain mechanical part and ask us to come up with a machining sequence for it. If you are not acquainted with machinist’s jargon, here’s a definition:

Definition: a machining sequence is a sequence of machining operations (such as turning, drilling, milling, boring, broaching, etc) that remove material from, say, an aluminium block in order to obtain a desired object (such as a piston or a cylinder for a car engine).

Note that machining is about removing material, i.e., it’s a subtractive technology. By contrast, rapid prototyping is an additive technology.

Let us consider an example. Suppose I give you the following 3D model:

What machining sequence would you propose? Here’s my proposal:

1. Start with a crude block of aluminium.
2. Given the object’s rotational symmetry about the longitudinal axis, use the lathe to shape the cylindrical / conical parts. Use the lathe also to drill the central hole along the longitudinal axis.
3. Note that the surface of the central hole is not smooth. It has “teeth”. I would say that a combined lathe / broach would allow one to obtain the “teeth”.
4. Drill 6 holes at the base.
5. Use a milling machine to remove material at the base, in between the 6 holes.

Here’s a sketchy illustration of the machining sequence I proposed, where a machine tool is assigned to various sections of the object being fabricated:

I am not an experienced machinist, and I make no claims that the machining sequence I have just proposed is any good. If you have ideas, feel free to share.

Suppose you have a 3D model of some object you would like to fabricate via machining. An experienced machinist is able to visualize the object and devise a machining sequence. Could a computer be taught to do the same?

Problem: develop algorithms that, given a 3D model of some object, automatically generate machining sequences. These algorithms should determine which machining operations to use, and in which order.

The machining sequence to be carried out is a function of the geometry and the topology of the object to be fabricated. Hence, algorithms that automatically generate machining sequences would have to “understand” the geometry and the topology of the given 3D model. This suggests that ideas from Computational Geometry and Computational Topology could perhaps be applied in this particular problem. For example, if the object to be fabricated has cylindrical holes in it, then it’s likely that one should use a drill or a lathe. These holes are a geometrical / topological property of the object. How can a computer be taught to understand them?

Furthermore, there probably is more than one possible machining sequence to fabricate the desired object from the same crude block of metal. If there are many possible machining sequences, then one can start thinking of optimality. Some possible optimality criteria would be to minimize the total number of operations, or to minimize the total fabrication time, for example. Can you think of other criteria? Given one optimality criterion, one can then try to find the “best” machining sequence.

I wonder if Machine Learning could help, too. Give the same 3D model to an experienced machinist and to the software system we’re developing. Both devise machining sequences. The human will probably beat the computer. Feed the computer with the machining sequence devised by the human. Repeat this procedure many times. Iterate until one has obtained a skilled “machining expert system“. The ingenuity and creativity required to transform crude blocks of metal into beautifully machined parts would be emulated by algorithms. I make no claims that this would be easy to implement ;-)

I looked for work on this area. I found a few patents [1]-[5], and a bunch of technical papers [6]-[12] dedicated to this problem. Interestingly, most patents I found on this topic were filed by Japanese. This should come as no surprise. After all, the Japanese are notoriously good at optimizing manufacturing processes.

This post is admittedly vague. I propose a problem, but I do not present any sketch of a solution. Basically, I wanted to share my thoughts in order to invite constructive discussion. Any ideas?

__________

References

[1] Hajimu Kishi, Masaki Seki, Kunio Tanaka, Teruyuki Matsumura, Automatic machining process determination method in an automatic programming system, U.S. patent #4723203. Filed: 1985 / Issued: 1988.

[2] Koichi Asakura, Machine tool with tool selection and work sequence determination, U.S. patent #4739488. Filed: 1985 / Issued: 1988.

[3] Yasushi Fukaya, Yuto Mizukami, Method for determining a machining method in numerical control information generating apparatus, U.S. patent #5107414. Filed: 1989 / Issued: 1992.

[4] Kotaro Watanabe, CAD/CAM unit data generating apparatus and process, U.S. patent #5297022. Filed: 1992 / Issued: 1994.

[5] Takashi Takegahara, Shigetoshi Takagi, Koji Suzuki, Machining sequence determining method and apparatus for wire-cut electric discharge machining and computer readable medium storing a machining sequence determining program, U.S. patent #6445972. Filed: 1999 / Issued: 2002.

[6] In-Ho Kim, Jung-Soo Oh and Kyu-Kab Cho, Computer aided setup planning for machining processes, Computers & Industrial Engineering, 18th International Conference on Computers and Industrial Engineering, Vol. 31, Issues 3-4, pp. 613-617, December 1996.

[7] Z. Gu, Y.F. Zhang, A.Y.C. Nee, Identification of important features for machining operations sequence generation, International Journal of Production Research, Vol. 35, Issue 8, pp. 2285-2308, August 1997.

[8] Majid Tolouei-Rad, An efficient algorithm for automatic machining sequence planning in milling operations, International Journal of Production Research, Vol. 41, Issue 17, pp. 4115-4131, November 2003.

[9] W.D. Li, S. K. Ong, A.Y.C. Nee, Optimization of process plans using a constraint-based tabu search approach, International Journal of Production Research, Vol. 42, Issue 10, pp. 1955-1985, May 2004.

[10] D. Kingsly Jeba Singh, C. Jebaraj, Feature-based design for process planning of machining processes with optimization using genetic algorithms, International Journal of Production Research, Vol. 43, Issue 18, pp. 3855-3887, September 2005.

[11] L. Wang, N. Cai, H.-Y. Feng, Z. Liu, Enriched machining feature-based reasoning for generic machining process sequencing, International Journal of Production Research, Vol. 44, Issue 8, pp 1479-1501, April 2006.

[12] Rezo Aliyev, A strategy for selection of the optimal machining sequence in high speed milling process, International Journal of Computer Applications in Technology, Vol. 27, Issue 1, pp. 72-82, September 2006.

To understand reality, you have to understand how things work. If you do that, you can start to do engineering with it, build things. And if you can’t, whatever you’re doing probably isn’t good science. To me, engineering and science aren’t separate endeavors.

-Carver Mead

Prof. Carver Mead (b. 1934) has made major contributions to solid-state electronics, such as his work on “electron tunneling” and building the first GaAs MESFET.

Here are a couple of interesting interviews with Carver Mead:

• Interview with Carver A. Mead (Caltech – July 17, 1996): in this interview Mead talks about his childhood in California, how he got interested in electrical apparatuses and radios at a tender age, his undergraduate years at Caltech, how he started working in solid-state electronics, fundamental research, technology transfer, etc.

• Carver Mead’s natural inspiration (Technology Review, Sept. 2004): Mead talks about science and engineering, his way of doing research, the startups he helped get off the ground, the business of technology, “demand pull” versus “technology push”, neuromorphic electronic systems, etc. A PDF copy of this interview can be found here.

__________

Related:

• Inventor of the week: Carver Mead

• Carver Mead: 1999 Lemelson-MIT Prize Winner

• Carver Mead: a trip through four eras of innovation (2004)

• Carver Mead: looking to the future (2006)

Contrary to popular belief, helium is not used only to inflate party balloons. Given that helium is inert, it is used to create nonreactive atmospheres in semiconductor fabrication and in arc welding, for instance. However, the most important industrial application of this noble gas is probably in cryogenics. Liquid helium is used to cool infrared sensors, nuclear reactors, and superconducting magnets (for example, in MRI equipment). Helium is also used for leak detection, and as a gain medium in He-Ne lasers.

Helium is the 2nd most abundant element in the universe, right after hydrogen. Helium is produced in stars by nuclear fusion:

[ source ]

On Earth, the helium-4 isotope is by far the most abundant, a byproduct of millions of years of alpha decay of uranium and thorium (note that the nucleus of helium-4 is the same as the one of an alpha particle). Helium is extracted during the refining of natural gas. If it’s not extracted during this refining process, it soars off and is forever lost.

Since nuclear fusion technology is still decades away and given that alpha decay is an extremely slow process, industrial production of helium is indeed very far from becoming a reality. We will need to keep extracting helium from natural gas reserves. The problem is that helium reserves may be depleted within the next decade [1][2]. Once that happens, where will we get the helium demanded by high-tech industries?

If helium becomes a sought-after commodity, then the same companies that extract natural gas will have an economic incentive to extract helium as well, instead of letting it escape to the atmosphere. However, helium is a non-renewable resource. Sooner or later, we will deplete the reserves that have built up over millions and millions of years.

The Sun produces colossal amounts of helium by nuclear fusion. In fact, helium is named after Helios, the Greek god who personified the Sun (helium was first found in the Sun). Harvesting helium from the Sun is impossible, however. At least directly. A possibility would be to harvest helium-3 (to serve as fuel for fusion reactors) from moon rocks [3][4][5], where it can be found due to billions of years of bombardment by the solar wind. If the idea of mining the Moon sounds insane to you, please note that Harrison Schmitt (geologist and Apollo 17 astronaut) is one of the proponents.

Is the Moon the El Dorado of helium-3? I don’t know, but I have heard rumors that India and China are developing robotic systems to harvest helium-3 from moon rocks. It seems that Space is the next frontier in the race for natural resources. What next? Will we be mining asteroids to extract platinum for fuel cells?

__________

References:

[1] Emily Jenkins, A Helium Shortage?, Wired Magazine, Aug. 2000.

[2] Laura Deakin, The coming helium shortage, Chemical Innovation, June 2001.

[3] Harrison Schmitt, Mining the Moon, Popular Mechanics, Oct. 2004.

[4] John Lasker, Race to the Moon for Nuclear Fuel, Wired Magazine, 2006.

[5] Mark Williams, Mining the Moon, Technology Review, Aug. 2007.

Posted in Uncategorized Tagged: Helium, Helium-3 ]]> http://stochastix.wordpress.com/2009/09/02/are-we-running-out-of-helium/feed/ 7 stochastix Deuterium-Tritium Fusion http://stochastix.wordpress.com/2009/09/01/dangerous-knowledge/ http://stochastix.wordpress.com/2009/09/01/dangerous-knowledge/#comments Tue, 01 Sep 2009 13:53:03 +0000 Rod Carvalho http://stochastix.wordpress.com/?p=4473 ]]>

Bring vor, was wahr ist.
Schreib so, dass es klar ist.
Und verficht’s, bis es mit dir gar ist.

-Ludwig Boltzmann

Dangerous Knowledge is a 90-minute long BBC documentary about the lives of four great thinkers: Georg Cantor (1845-1918), Ludwig Boltzmann (1844-1906), Kurt Gödel (1906-1978), and Alan Turing (1912-1954). Cantor founded Set Theory, Boltzmann founded Statistical Mechanics, Gödel’s incompleteness theorems had an immense impact on Mathematical Logic, and Turing was one of the fathers of Computer Science. Why these four scientists? What is the pattern? The answer is that all four of them committed suicide.

The documentary is a bit too sensationalist for my taste. Its message is that these four brilliant minds were driven to madness (and, ultimately, to suicide) due to the earth-shattering nature of their ideas. While one could argue that such a claim is not too far-fetched in the cases of Cantor and Boltzmann, it seems somewhat distasteful when applied to Turing. Turing did not go insane because of the depth of his revolutionary insights. Turing’s homosexuality was viewed as a security problem during the troubled times of the Cold War, and the brutal punishment which he underwent (chemical castration) was probably what led him to take his life. Gödel feared that someone was poisoning his food, and would refuse to eat unless his wife would taste his food first. When his wife was hospitalized for some time, Gödel starved himself to death. Hence, the claim that these four suicides were due to “dangerous knowledge” seems rather crude to me.

On the bright side, the claim that these four geniuses were ahead of their time sounds rather plausible. Cantor’s or Boltzmann’s ideas did not meet fierce resistance because they were technically wrong, but because they challenged the belief that the universe was perfect and orderly. No one wanted to hear about Boltzmann’s work on entropy, because entropy is, by definition, a measure of disorder. As Hilbert’s program tried to fix the inconsistencies in  the foundations of Mathematics, Gödel’s incompleteness theorems made Hilbert’s monumental mission “wir müssen wissen, wir werden wissen” seem even harder to accomplish.

In my most humble opinion the times were not yet ripe for theories that imposed limitations on human knowledge. “What about Heisenberg’s uncertainty principle?”, I hear you ask. Good point. Do note that two decades can make a world of difference. Heisenberg’s work was created when Quantum Mechanics was already out of the box. By contrast, Boltzmann had to fight the 19th Century establishment that was still deeply entrenched in the Newtonian paradigm.

If you have a couple of hours, here are the documentary’s videos:

[ part 1 | part 2]

Greg Chaitin and Sir Roger Penrose are included in this documentary, which itself makes it worth watching.

Welcome to the 56th installment of the Carnival of Mathematics!

Qiaochu Yuan writes about Kraft’s inequality for prefix codes and the probabilistic method, and also on linear relationships between square roots.

Prolific blogger John Cook writes about an intriguing nonlinear differential equation , with initial value . Suppose we would like to compute . Numerical methods produce rather bizarre results. The reason is that there is no solution at (finite escape time, anyone?). The moral of the story is that the theorems on existence and uniqueness of solutions to initial value problems do indeed matter.

Rick Regan discovered that positive integers of the form have binary representations where the least significant bits are exactly . For example, . It’s interesting to note that this was a somewhat serendipitous finding.

Harrison Brown has a post on Roth’s proof of Roth’s theorem on arithmetic progressions from a combinatorial viewpoint.

Dave Richeson writes about the prime , twin primes and Pythagorean triples.

Américo Tavares lists three gamma function identities.

Mike Croucher writes about Philipp Kühl and Daniel Kirsch’s awesome Detexify project, a symbol classifier. This is not mathematics per se, but it is an interesting tool for those who need to type mathematical expressions. And if you are interested in machine learning, you will definitely be intrigued.

Since there were few submissions to this Carnival, I take the liberty of including some more links of interest to me:

• Math Alive was a course on mathematical concepts that have changed the world. It covers many interesting topics: Cryptography, Error Correction Codes, Data Compression, Probability and Statistics, Chaos and Complex Systems, Graph Theory, Voting and Social Choice Theory. Take a look at the lecture notes. The course was taught at Princeton University by Ingrid Daubechies.

• Edsger Dijkstra once said that “Computer Science is no more about computers than Astronomy is about telescopes”. Computer Science Unplugged is a very interesting initiative whose aim is to teach the principles of Computer Science via a series of recreational learning activities that are suitable for kids of all ages.

• Harold Kuhn’s lectures on the theory of games are a delightful read!

• If you are interested in Combinatorial Game Theory, you have to read Games of No Chance.

• Igor Pak’s Lectures on Discrete and Polyhedral Geometry [pdf] look terribly sexy indeed.

• James Sethna’s gorgeous book Statistical Mechanics: Entropy, Order Parameters, and Complexity is available online.

• Gil Kalai wrote on strategy-proof mechanisms and an impossibility theorem. Fascinating stuff!

• Ron Doerfler wrote three beautiful posts on the lost art of Nomography. If you like to write code in Python, you can create your own nomographs with PyNomo. If this looks pre-historical to you, be reminded that electrical engineers (such as myself) still use Smith charts :-P

• David Austin wrote an awesome introduction to the Singular Value Decomposition. He also wrote a really good article on Percolation.

• Reading Brad Osgood’s The Fourier Transform and its Applications [pdf] is a great way of relearning Applied Fourier Analysis.

• If you are studying or teaching elementary Game Theory, give Gambit a try!! Human beings were not designed to compute Nash equilibria with pencil and paper…

• A very interesting applied problem I recently read about is the German tank problem. During WW2, the allies wanted to estimate the number of tanks the Germans were producing each year, so that they could assess the likelihood of success of an invasion of Continental Europe. Intelligence reports were not conclusive. The allies’ statisticians were able to estimate with great precision the number of German tanks being produced from the serial numbers of captured German tanks. The fact that the Germans, in typical German fashion, numbered their tanks sequentially turned out to be quite a blunder ;-)

• Here’s something for the aesthetically-inclined: visualizing the 4th dimension using color.

• If you are interested in vintage computers, you might want to take a look at Sketch of the Analytical Engine (1842).

• Erico Guizzo wrote a thesis on Claude Shannon and the making of Information Theory.

• You can build a Tinkertoy computer that plays tic-tac-toe.

• If you happen to suffer from insomnia, I do recommend the truly monumental Principia Mathematica.

• If you like Calvinball, then give Nomic a try!!

The next edition of the CoM will be posted in two weeks, on 360.

Matthias Wandel built a rudimentary digital computer out of wood. The ingenious Binary Marble Adding Machine is, basically, a 6-bit adder that runs on gravity and uses mechanical flip-flops for memory storage.

[ schematic courtesy of Matthias Wandel ]

Here’s a demo video:

More info on the lovely Marble Adding Machine:

• More on the binary marble adding machine

• Plans for the marble adding machine

• How the marble flip-flop works

Other interesting creations of Matthias Wandel:

• Wooden Machines

• Marble Machines

(hat tip: Rick Regan)

Consider a network of agents, each owing money to and being owed money by other agents. The debts are represented pictorially by the weighted directed graph

which I called an IOU graph on my previous post on this topic. The IOU graph representing this network is an ordered triple , where is the node set (each node of the IOU graph represents a different agent), is the edge set (containing elements), and is a weight function that assigns positive weights to each edge in . Each directed edge of graph is an ordered pair , where is the edge’s tail and is the edge’s head. If then node owes money to node . We do not allow nodes to owe money to themselves and, therefore, for each we must have . The weight of edge is given by , and it tells us the amount of money that node owes to node .

Alternatively, the information contained in the IOU graph can be encoded in the following nonnegative matrix

where entry is the amount of money node owes to node . Since we do not allow nodes to owe money to themselves, the entries on the main diagonal of this matrix are zero. We will call this matrix the IOU matrix. Note that the IOU matrix is the adjacency matrix of the IOU graph. Every nonnegative matrix with zero entries on the main diagonal is an admissible IOU matrix, and we can associate with it a corresponding IOU graph.

If the IOU matrix contains all the information about the IOU graph, then we can work with the IOU matrix instead of the IOU graph. Is there any advantage in doing so? I believe there is. In my opinion, the IOU graph is good for conceptual thinking and intuition, while the IOU matrix is a data structure that is convenient for computation. However, the matrix is not the only data structure that one can use to encode the information in an IOU graph. Another possibility would be to use adjacency lists, for example.

__________

Divergence and IOU Matrices

Suppose we are given an IOU graph , whose node set is . Note that for each , because nodes cannot owe money to themselves. We now introduce variables as follows

and build the (nonnegative) adjacency matrix of the IOU graph

which is the IOU matrix associated with the IOU graph . The entry of matrix denotes the amount of money that node owes to node . The divergence of node is a scalar defined as

and it gives us the amount of money node owes to other nodes minus the amount of money node is owed to by other nodes. In terms of the IOU matrix , the divergence of node is given by

where is the -th vector of the canonical basis for , i.e., the -th column of the identity matrix , and is an -dimensional vector of ones. Let denote the vectorization operator, which stacks a given matrix’s columns in one large column vector. Note that if we apply operator to a scalar, we obtain the same scalar. Hence, we have that

and

where denotes the Kronecker product. Therefore, the divergence of node can be written as

.

We define also the divergence vector of graph as the -dimensional vector whose -th component is

.

Alternatively, we can use equation , and write the divergence vector as

and, since

and

,

we obtain

.

Note that -dimensional vectors and contain the same entries, but in a different order. Hence, there exists a permutation matrix such that

which allows us to write

.

Do note that the permutation matrix depends on only. It does not depend on the entries of the matrix to which operator is applied. Henceforth, we will denote . Finally, we have a rather compact equation for the divergence vector

.

The divergence vector of an IOU graph tells us how much money each node in the graph will lose after debts have been paid. We say that two IOU graphs (on the same node set) are equi-divergent if their divergence vectors are the same. This means that, although the debt relationships are different, both IOU graphs result in every node getting paid what he is owed and paying what he owes.

__________

Minimum Money Flow

Suppose we are given an IOU graph . We would like to find a new IOU graph on the same node set , such that the two IOU graphs are equi-divergent, i.e., . Moreover, we would like the new IOU graph to be “optimal” in some sense. On this post, we will consider the following optimality criterion:

minimum money flow: we want to minimize the total amount of money being exchanged between the nodes. This is the same as minimizing the sum of the weights of the IOU graph .

Let us start with matrix , which will be an admissible IOU matrix if we impose two constraints:

• nonnegativity: the entries of matrix must be nonnegative. This is equivalent to imposing the constraint that vector has nonnegative entries, i.e., , which is equivalent to .

• zero entries on the main diagonal: we must have for all . Note that , and therefore is equivalent to . Hence, imposing the constraint that the entries on the main diagonal be zero is equivalent to imposing the equality constraints

.

Once these two constraints have been imposed, matrix is the adjacency matrix of an IOU graph. Finally, we must impose the constraint that the divergence vector of the new IOU graph is the same as the divergence vector of the given IOU graph, i.e., , which yields

where can be computed from the given IOU graph. Considering the minimum money flow criterion, the cost function to be minimized is the following

and, hence, we obtain the following linear program in

which has equality and inequality constraints. One can easily solve this linear program with MATLAB using function linprog.

However, this formulation looks a bit silly. Note that initially we have degrees of freedom (the entries of matrix ), then we impose equality constraints to ensure that the entries on the main diagonal are zero, which leaves us with degrees of freedom. If we already know that the entries on the main diagonal are zero, why consider them as decision variables? Wouldn’t it make much more sense to consider only the entries off the main diagonal as decision variables?

Let be a binary matrix such that is the reduced vector of decision variables, containing solely the entries of off the main diagonal. We now eliminate the columns of matrix which were to be multiplied by the variables. We do so by multiplying matrix on the right by , which produces a matrix .

The new vector of decision variables is . Since we no longer consider the entries on the main diagonal as decision variables, we don’t need to impose the equality constraints . Hence, we obtain a lower-dimensional linear program

where we now have equality constraints (divergences at the nodes), and inequality constrains (). My intuition tells me that

is an incidence matrix of the directed complete graph on node set . What do you think? Do you agree?

__________

A Numerical Example

Consider a network of agents whose debts are represented by the following IOU graph (with edges)

The IOU matrix associated with this given IOU graph is

Solving the linear program, we obtain a new IOU matrix

which is the adjacency matrix of a new IOU graph. The given IOU graph and the new IOU graph are equi-divergent. The given IOU graph had edges and a total of flowing in it, while the new IOU graph has edges and a total of flowing in it. While the flow of money has been reduced, the number of edges (i.e., the number of transactions) has increased dramatically!!

On my previous post on this topic, I gave two very simple examples, and in both of them the minimum money flow solution also yielded the least number of transactions. I wondered back then if that was always the case. This example clearly shows that it is not. Generally speaking, not only does the minimum money flow criterion fail to minimize the number of transactions, but it can actually result in more transactions than the ones in the initial IOU graph. I cannot think of any reason why minimizing the total amount of money flowing in the IOU graph at the cost of a larger number of transactions would be a good idea…

Therefore, the next step is to abandon the minimum money flow criterion, and focus solely on the least number of transactions. Minimizing the number of transactions is a hard combinatorial problem, but maybe there are approximation algorithms that make the problem tractable.

If you would like to use POV-Ray to visualize 3-dimensional geometrical objects, I highly recommend Friedrich A. Lohmueller’s beautiful tutorial on Analytical Geometry with POV-Ray. Other POV-Ray tutorials by Friedrich A. Lohmueller can be found here.

[ image courtesy of Friedrich A. Lohmueller ]

In my humble opinion, this is how Analytical Geometry should be taught. Descartes had no access to 3-d graphics. We do. Why not take advantage of the technology?

Andrea Montanari and Marc Mézard have written a book: Information, Physics and Computation. It is a unique, ambitious book which covers the exciting intersection of Statistical Physics,  Theoretical Computer Science, Discrete Mathematics, Information Theory, and Coding Theory.

[ graphs courtesy of Montanari & Mézard ]

The book was published recently. Hopefully, the PDF files will continue to be available online. If you liked David MacKay’s gorgeous book, you will probably like this one too.

I remember on the trip home on Apollo 11 it suddenly struck me that that tiny pea, pretty and blue, was the Earth. I put up my thumb and shut one eye, and my thumb blotted out the planet Earth. I didn’t feel like a giant. I felt very, very small.

-Neil Armstrong [1]

On this day 40 years ago, the Apollo 11’s Eagle lunar module landed on the Moon. As Neil Armstrong and Buzz Aldrin became the first humans to walk on the lunar surface, Michael Collins orbited the Moon inside the Columbia command module, experiencing “an aloneness unknown to man before”, as aviator Charles Lindbergh put it [2].

[ photo courtesy of NASA ]

The Apollo program was an enormous technical achievement. I shall forever be amazed at how the engineers who worked on this program managed to accomplish so much with the technology they had available at the time. The rocket engine was a relatively recent technology back then, the transistor had been invented in 1947, the integrated circuit had been invented in 1958, and the microprocessor had not even been invented yet. In particular, the Apollo Guidance Computer (AGC) was a wonderful work of ingenuity: it was the world’s first modern real-time embedded system, and it led to the development of fly-by-wire systems.

To commemorate the 40th anniversary of the Apollo 11 mission, the command module code (Comanche054) and the lunar module code (Luminary099) have been transcribed from scanned images by the Virtual AGC and AGS project! [3] The code can be found here.

For example, take a look at the following modules: ascent guidance, Kalman filter, master ignition routine. Yes, programming the AGC seems to have been a spartan endeavor! ;-) If you happen to dislike vintage assembly programming languages, take a look at the yaAGC code, which is written in C.

__________

References:

[1] The Greening of the Astronauts (Dec. 1972)

[2] How Michael Collins became the forgotten astronaut of Apollo 11 (July 2009)

[3] Apollo 11 mission’s 40th Anniversary: One large step for open source code… (July 2009)

When I first studied Chemistry, I learned that there are two allotropes of carbon: graphite and diamond. As you know, these two forms of pure carbon have dramatically different mechanical, electrical, optical and thermal properties. Then, in the early 1990s, I read with excitement about a new form of pure carbon, carbon-60, which consists of 60 carbon atoms bonded together in a polyhedral structure which resembles a soccer ball:

[ image source ]

This novel molecule was named Buckminsterfullerene (after Buckminster Fuller), and is colloquially called buckyball. The beautiful carbon-60 molecule was discovered in the mid-1980s at Rice University by Robert F. Curl Jr., James R. Heath, Sir Harold W. Kroto, Sean C. O’Brien, and Richard E. Smalley by vaporizing carbon with a laser beam and allowing the carbon atoms to reconstitute in clusters with a “soccer-ball-like” structure. The following documentary (divided in 5 parts) tells the exciting story of this discovery:

[ part 1 | part 2 | part 3 | part 4 | part 5 ]

In 1996, Bob Curl, Harry Kroto and Rick Smalley won the Nobel prize in Chemistry for the discovery of carbon-60.

Hat tip: Shubhendu Trivedi

__________

Related:

• Art, buckyballs and chemistry: the ABCs of discovery. A conversation with Nobel Laureate Sir Harold Kroto (Nov. 2, 2007)

Suppose we are given red balls and black balls. All of these balls are labeled with a real number. For example

We now pair red balls with black balls. There are such matchings, as illustrated below

Additionally, suppose that a pair composed of a red ball with label and a black ball with label can be exchanged for a white ball with label . For example

Since there are matchings, in general there will be collections of white balls, as depicted below

Note that the sum of the labels of the white balls is always , regardless of the matching. This is due to the fact that addition is commutative and associative. Note also that the resulting collections of white balls are not necessarily distinct.

Suppose now that we would like to allocate the white balls to bowls such that the sum of the labels of the white balls in each bowl is 50% of the total sum of the labels of the white balls. Pictorially, we have

and, since the sum of the labels of the white balls in each bowl is exactly , the allocation is feasible. Nonetheless, this allocation is not unique: we could also have allocated ball with label 3 to bowl 1 and balls with labels 1 and 2 to bowl 2. If we wanted to allocate 1/3 of the total to bowl 1 and 2/3 of the total to bowl 2, we could have allocated ball with label 2 to bowl 1, and balls with labels 1 and 3 to bowl 2, for instance. This allocation would also be feasible.

However, not all desired allocations are feasible. For example, suppose that we want to allocate 90% to bowl 1 and 10% to bowl 2. This allocation is unfeasible. We can allocate a total of 5 to bowl 1 and 1 to bowl 2, which corresponds to 83.3% and 16.7% instead of 90% and 10%.

Though we cannot attain the desired allocation exactly, we are able to get relatively “close” to it. We could use, for instance, a quadratic criterion to measure how “far” a particular allocation is from the desired one.

In each bowl we are attempting to reconstruct a given real number from a finite collection of real numbers. Hence, the unfeasibility of a desired allocation stems from the “chunkiness” of the “parts” we have at our disposal. But here is the interesting aspect of this problem: we can change the “parts” (i.e., the white balls) by matching the red and black balls in a different way. Therefore, allocation can not be “decoupled” from matching, so to say. The two operations are now intertwined. When performing the matching, we should strive to build a good collection of white balls so that the actual allocation is as “close” as possible to the desired one.

If we have red balls and black balls, then there are matchings. Each red-black pair is then allocated to one of bowls. Hence, our search space has points, and an exhaustive search will run in time which is intractable for “large” values of . For example, if and , the search space has over billion points! Yet once again, we are doomed by the curse of combinatorial explosion.

Our challenge (and only hope) is to design a heuristic-based algorithm which, although not optimal, is “good enough” most of the time. Any ideas?

This problem seems to have some of the ingredients of many classical problems such as integer partitions, knapsack, and bipartite / tripartite matching. It is not difficult to formulate the problem as a constrained quadratic binary optimization problem, but solving it is hard.

Suppose that Alice owes Bob $5 and is owed $10 by Bob. These debts can be represented as the weighted directed graph

which we will call an IOU graph. Alice pays Bob $5 and Bob pays Alice $10. The debts are paid and $15 have been exchanged in a total of two transactions. Let us now consider the following cash flow diagram

and, accounting for the net cash flows, we obtain

which illustrates that if Bob paid Alice $5, then the debts would still be paid, though only $5 would be exchanged and only one transaction would be made. This looks much cleverer and more efficient than exchanging $15 in a total of two transactions. Hence, the following IOU graph

is “equivalent” to the original IOU graph. In both IOU graphs, if debts are paid then Alice will be $5 up, while Bob will be $5 down.

We add a third agent, Charlie. Consider now the IOU graph

where:

• Alice must pay a total of $20 and be paid a total of $30.

• Bob must pay a total of $15 and be paid a total of $20.

• Charlie must pay a total of $35 and be paid a total of $20.

Hence, after the debts are paid, Alice will have $30 – $20 = $10 more in her account, Bob will have $20 – $15 = $5 more in his account, and Charlie will have $20 – $35 = -$15 more (= $15 less) in his account. Note that $10 + $5 – $15 = $0, i.e., money is conserved. In total, $70 have been exchanged in six transactions. We can do better than that. For example, we could pay all debts in just three transactions, as illustrated in the following IOU graph

where a total of $20 are exchanged. Note that Alice is $10 up, Bob is $5 up, and Charlie is $15 down, just like before. We can do even better than this. If Charlie pays Bob’s debt to Alice, then all debts can be paid in just two transactions as shown in the IOU graph below

where only $15 are exchanged. This is an “optimal” debt-paying scheme because the debts cannot be paid in less than two transactions, nor exchanging less than $15. Do you agree?

__________

Debts and IOU Graphs

An IOU graph is a weighted directed graph given by an ordered triple , where:

• is the node set. Each node of the IOU graph represents a different agent.

• is the edge set. Each directed edge is an ordered pair , where is the edge’s tail and is the edge’s head. If then node owes money to node . We do not allow nodes to owe money to themselves and, therefore, for each we must have .

• is a weight function that assigns positive weights to each edge in . The weight of edge is given by , and it tells us the amount of money that node owes to node .

We define the in-neighborhood and out-neighborhood of node as follows

and illustrate them below

The divergence [1] of node of graph is a scalar defined as

and it gives us the amount of money node owes to other nodes minus the amount of money node is owed to by other nodes. The divergence of a node can be viewed as the discrete, graph-theoretic analogue of the divergence operator in vector calculus. We define also the divergence vector of graph as the vector whose -th component is

.

The divergence vector of an IOU graph tells us how much money each node in the graph will lose after debts have been paid. We say that two IOU graphs (on the same node set) are equi-divergent if their divergence vectors are the same. This means that, although the debt relationships are different, both IOU graphs result in every node getting paid what he is owed and paying what he owes.

__________

Optimal Account Balancing

Suppose we are given an IOU graph . We would like to find a new IOU graph on the same node set , such that the two IOU graphs are equi-divergent

.

Moreover, we would like the new IOU graph to be “optimal” in some sense. I can think of the two following optimality criteria:

minimum money flow: we want to minimize the total amount of money being exchanged between the nodes. This is the same as minimizing the sum of the (positive) weights of the edges of the IOU graph.

least number of transactions: we want to minimize the total number of transactions being made. This is the same as minimizing the number of edges in the IOU graph, i.e., maximizing the sparsity of the IOU graph.

Can you think of other interesting optimality criteria?

It would be reasonable to consider the case where the transaction fees are nonzero. If these are variable and proportional to the amount of money being transferred, then we would like to minimize the amount of money flowing in the network of agents. If the transaction fees are fixed, then we would like to minimize the total number of transactions being made. For example, in the two examples above the minimum money flow solution is the same as the least number of transactions solution. Is this always the case?

In order to find the optimal IOU graph, we start with a weighted directed graph on node set whose edge set is the one of the directed complete graph on . Hence, every two distinct nodes are connected by two directed edges with opposite directions. The weight of edge is denoted by . Since some of these weights might be zero, there might be edges in this graph which do not represent debt relationships! Therefore, although this graph is weighted and directed, it is not necessarily an IOU graph. We optimize for the nonnegative variables and, in the end, we remove the edges whose weight is zero so that we obtain an IOU graph. I view this approach as somewhat akin to scaffolding ;-)

For example, considering the minimum money flow criterion, we have the following minimum cost flow problem [1]

which can be solved via linear programming [2]. In words: we want to find the debts such that all debts are paid (each node should be paid what it is owed minus what it owes to other nodes) and the total amount of debt is minimized, which results in the total amount of money flowing to be also minimized.

The least number of transactions criterion is much harder to tackle because we want to maximize the sparsity of the graph, while ensuring that the graph and the original IOU graph equi-divergent. We want to optimize the topology of the graph and, hence, a combinatorial flavor is added to the problem. I will write more about it on future posts.

__________

A Numerical Example

Let us again consider the -agent example where the agents are Alice, Bob and Charlie. The IOU graph is

where the node set is . We introduce vector

where is the amount of money node owes to node . The divergence of the new graph at each node must be the same as the divergence of the given IOU graph at each node, which yields the equality constraint , where

is the incidence matrix of the graph, and is the opposite of the divergence vector of the given IOU graph above. This equality constraint guarantees that the given IOU graph and the new graph we obtain are equi-divergent.

Let be the identity matrix, let be the -dimensional vector of ones, and let be the -dimensional vector of zeros. The nonnegativity constraints can be written in matrix form as . Considering the minimum money flow criterion, we obtain the linear program

Note that the equality constraint is equivalent to two inequality constraints: and . Thus, we can transform the linear program above into a linear program with inequality constraints only

If you have MATLAB, you can easily solve these linear programs using function linprog. The following Python 2.5 script solves the latter linear program using CVXOPT:

from cvxopt import matrix
from cvxopt import solvers

# -------------------
# auxiliary matrices
# -------------------

# defines incidence matrix
C = matrix([ [-1.0, 1.0, 0.0],
             [-1.0, 0.0, 1.0],
             [1.0, -1.0, 0.0],
             [0.0, -1.0, 1.0],
             [1.0, 0.0, -1.0],
             [0.0, 1.0, -1.0] ])

# defines desired divergence vector
d = matrix( [-10.0, -5.0, 15.0] )

# defines 6x6 identity matrix   
Id = matrix([ [1, 0, 0, 0, 0, 0],
              [0, 1, 0, 0, 0, 0],
              [0, 0, 1, 0, 0, 0],
              [0, 0, 0, 1, 0, 0],
              [0, 0, 0, 0, 1, 0],
              [0, 0, 0, 0, 0, 1] ])

# ---------------
# linear program
# ---------------

# defines objective vector
c = matrix(1.0, (6,1))

# defines inequality constraint matrices
G = matrix( [C, -C, -Id] )
h = matrix( [-d, d, matrix(0.0, (6,1))] )

# solves linear program
solvers.options['show_progress'] = True
solution = solvers.lp(c, G, h)

# prints solution
print solution['x']

The solution is

,

which is “contaminated” with numerical noise. Adjusting the tolerances of the numerical solver, one can reduce the numerical noise and obtain the true solution, which is . Removing the edges with zero weight, we obtain an IOU graph with 2 edges

and, hence, only two transactions are required, and only $15 need to be exchanged. This is the exact same solution we obtained previously via heuristic methods! :-)

__________

Concluding Remarks

The two optimal debt-paying schemes considered on this post rely on an “accountant” who has perfect global information about all the debts in the network of agents. This accountant solves an optimization problem to find out what transactions should be carried out. This is a centralized approach, as the decision power is centralized in the accountant. Another (and arguably more interesting) approach would be a decentralized one, where the nodes have only local information (i.e., each node only knows who owes him money and to whom he owes money) and where each node solves a local optimization problem using, for example, message-passing algorithms. I will write about it on future posts.

The only modeling of this problem I have found was on Demetri Spanos‘ PhD thesis [3]. If you know of any papers on this topic, please let me know.

Last but not least, allow me to add a personal note: I first started thinking of this problem years ago, while going to dinners with friends and trying to figure out how to split the bill in an efficient manner. The real world is a great source of inspiration when it comes to interesting problems.

__________

References

[1]  Dimitri Bertsekas, Network Optimization: Continuous and Discrete Models, Athena Scientific, 1998.

[2] Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge University Press, 2004.

[3] Demetri Spanos, Distributed gradient systems and dynamic coordination, Ph.D. thesis, California Institute of Technology, December 2005.

[ source ]

Let us start with the following joke:

Two business partners asked their lawyer to hold $20,000, making him promise to get both of their signatures before disbursing any of it. As soon as one partner left town, the other pressed the lawyer for $15,000, citing an emergency. The lawyer reluctantly gave it to him, and he disappeared. On his return, the other partner was irate, so the lawyer explained that he had donated the $15,000 out of his own pocket.

“Then give me the $20,000 you’re holding,” said the partner.

“All right,” said the lawyer. “Give me the two signatures.”

__________

These two business partners and their lawyer should have signed a contract. If the lawyer disbursed any of the partners’ money without having both of their signatures, he would face legal action for breach of contract. However, this contract would still not be immune to collusion: one of the two partners could split the money with the lawyer and disappear, and the other partner would never have the two signatures required by contract to have his money back. How would you design a better contract? By “better contract”, I mean:

• a contract that would NOT give any of the parties involved an incentive to break it.

• a contract that would be robust (and, ideally, immune) to collusion.

I make no claims that such a “perfect contract” even exists, but maybe one could devise a contract that would be better than the aforementioned one. Purely for recreational purposes, I propose a challenge: try to design the best contract you can think of!

If you have any ideas, please feel free to share them.

The Radiology Art project was started by artist and medical student Satre Stuelke in the Summer of 2007. Stuelke uses a CT scanner to acquire DICOM images of various objects (e.g., electronic apparatuses, toys) and then processes them with OsiriX Imaging Software and Adobe Photoshop. Colors are assigned based on the varying densities of materials present throughout the object. The post-processed images are stunning.

For example, here’s a CT scan of a Teslovak KT88S vacuum tube:

[ image courtesy of Satre Stuelke ]

Related:

• A Cultural Scan

• Diagnostic Art

In the days of my almost infinitely prolonged adolescence, I hardly took an action, hardly did anything that did not arouse in me a very great sense of revulsion and of wrong. My feeling about myself was always one of extreme discontent.

- J. Robert Oppenheimer (1904-1967)

The Trials of J. Robert Oppenheimer is a two-hour documentary film on the life of the brilliant and charismatic American nuclear physicist J. Robert Oppenheimer (1904-1967), from his youth and emergence as one of America’s leading physicists, to his leadership of the Manhattan Project, and his most tragic humiliation during the anti-Communist witch-hunt in the 1950s.

The film was produced by David Grubin, and it features actor David Strathairn as Oppenheimer. You can watch it online for free. The transcript is also available.

__________

Related:

• Shubhendu Trivedi’s review of American Prometheus

• Aurélie Thiele’s review of American Prometheus

• Robert Oppenheimer: 104th birthday (by Luboš Motl)

• Oppenheimer (by E. Kowalski)

Consider a finite set with distinct elements which we will call symbols. We will call set an alphabet. A word (also known as string) of length over set is formed by constructing an -tuple of symbols from alphabet

where for all . A string of length will be called an -string. The set of all -strings is given by the cartesian product

.

For example, in coding theory one usually works with binary words, i.e., words over alphabet . The set of all binary words of length is , whose cardinality is .

In this post we will work only with words over the alphabet

which is known as the Latin alphabet, and whose 26 symbols are known as letters. For example,  and are represented by the 3-strings and , respectively, where .

__________

Hamming distance

Given two strings of equal length, the Hamming distance between them is the number of positions for which the corresponding symbols are different. In other words, the Hamming distance between two strings of equal length is the minimum number of symbol substitutions required to change one string into the other.

Let function be defined as

where function is defined as

then is the Hamming distance between -strings and . For example, if and , then we have because and only differ in one symbol (the last one).

Suppose we are given an -string over the Latin alphabet . If we replace the letter at position with another letter in , where , we can obtain a total of new -strings, each within Hamming distance of the original -string. Let us call these -strings neighbors of the original -string. Thus, an -string has a total of neighbors.

__________

Word graphs

If two -strings are neighbors of one another when their Hamming distance is equal to , then a graphical approach would be the natural way to go. Let be an undirected graph whose set of nodes is (set of all -strings over alphabet ), and whose set of edges is . Two nodes are connected by an edge if their Hamming distance is equal to . Hence the total number of nodes in is , and the total number of edges in is

,

because each node has neighbors, there are nodes, and each edge is counted twice so we need to divide the total by . Note that is a regular graph (of degree ) because all nodes have the same number of neighbors.

For example, let and let us consider the -string , which has neighbors. Out of these words, I could count words which actually mean something in the English language:

bar, ear, far, gar, jar, mar, oar, par, tar, war, yar, cor, cur, cab, cad, cam, can, cap, cat, caw, cay.

Please let me know if I forgot to include any word in this list. These words form a sub-graph of the graph of all -strings. Note that the words

bar, car, ear, far, gar, jar, mar, oar, par, tar, war, yar

are all neighbors of one another, and hence, they form a clique of size . The words

car, cor, cur

form a clique of size . The words

cab, cad, cam, can, cap, car, cat, caw, cay

are all neighbors of one another as well, and thus they form a clique of size . The subgraph formed by the word “car” and its neighbors which make sense in the English language is illustrated below, where the three cliques are easy to identify

[ graph generated with Graphviz ]

It is not practical to depict the whole graph of -strings over alphabet since it contains nodes and edges!

__________

Python code

In case you are wondering: no, I did not type the 26 symbols of the Latin alphabet in . That would have been way too cumbersome. All it took was one single line of Python code:

", ".join([("\text{" + chr(i) + "}") for i in range(ord('a'),ord('z')+1)])

Suppose we would like to find all strings within Hamming distance of of string . The following Python script automates the process:

def MutateString(s, alphabet, index):
    li = []
    for ch in alphabet:
        if ch != s[index]:
            li.append(s[:index] + ch + s[index+1:])
    return li

# build Latin alphabet (lowercase symbols only)
Sigma = [chr(i) for i in range(ord('a'),ord('z')+1)]

# obtain list of mutated strings
print "Mutate 1st symbol:\n" + ", ".join(MutateString('cat', Sigma, 0)) + '\n'
print "Mutate 2nd symbol:\n" + ", ".join(MutateString('cat', Sigma, 1)) + '\n'
print "Mutate 3rd symbol:\n" + ", ".join(MutateString('cat', Sigma, 2)) + '\n'

The graph of the word “car” and its neighbors was generated also by a Python script using the GvGen library.

def HammingDistance(s1, s2):
    """Computes the Hamming distance between strings s1 and s2"""
    assert len(s1) == len(s2)
    return sum(ch1 != ch2 for ch1, ch2 in zip(s1, s2))

import gvgen

# list of words
words = ['car', 'bar', 'ear', 'far', 'gar', 'jar', 'mar', 'oar', 'par', 'tar', 'war', 'yar', 'cor', 'cur', 'cab', 'cad', 'cam', 'can', 'cap', 'cat', 'caw', 'cay']

# creates the new graph instance
graph = gvgen.GvGen()
graph.smart_mode = 1

# creates graph nodes
nodes = []
for elem in words:
    nodes.append(graph.newItem(elem))

# creates graph edges
for i in nodes:
    for j in nodes:

        # extracts the labels of each node
        node1 = i['properties']['label']
        node2 = j['properties']['label']

        # links nodes i and j if the Hamming
        # distance between their labels is 1
        if HammingDistance(node1,node2) == 1:
            graph.newLink(i,j)

# outputs the Graphviz code to a .dot file
filename = "test.dot"
FileOut = open(filename,"w")
graph.dot(FileOut)
FileOut.close()

The script outputs a DOT file which can be read by Graphviz. In particular, I used GVedit (with layout engine = circo) to generate the diagram depicted above.

Remark: these snippets of code worked with Python 2.5.2. I can not guarantee that they work with other versions.

Fotoopa is a photographer from Belgium who has shot amazing high speed photos of dancing and splashing colored water drops. Here’s an example of a droplet captured in mid-splash in vivid detail:

[ photo courtesy of fotoopa ]

Apart from the standard photographic equipment (camera, lenses, multiple flashes), fotoopa uses some high-tech hardware more commonly found on a physics research lab: lasers, LEDs, homemade electronics, power amplifiers, microcontrollers, FPGAs. It’s very, very impressive indeed.

Suppose we are given two lines in , and . Line passes through the point and its direction is given by vector

.

Line passes through the point and its direction is given by vector

.

What is the distance between lines and ?

__________

Lines as functions

Let vector-valued functions and be defined as

.

Lines and are thus the images of set under functions and , respectively

.

Function maps each into point . Therefore, choosing a particular value of corresponds to choosing a point on line , and vice versa (because function is injective). Thus, each point on the line can be unambiguously specified by a value of . Similarly, following the same line of thought, each point on the line can be unambiguously specified by a value of .

__________

Distance between two points

In order to compute the distance between lines and , we first need to know how to compute the distance between two points in , i.e., we need to choose a metric on . For the sake of simplicity, we will use the Euclidean metric, i.e., we will use the 2-norm to measure lengths and distances. In other words, we will be working in normed vector space .

Let us fix parameters and , i.e., and , and compute the distance between fixed points and

.

Pictorially, we have that is the length of the line segment connecting fixed points and

To each pair of parameters , it is assigned a non-negative scalar

.

__________

Distance between a point and a line

Suppose now that we vary , while remains fixed, . As we vary , we travel along line and hence the distance between free point and fixed point is given by

.

Since is now free, is a function of .

For example, let us pick three distinct values for , say . The image below depicts the line segments connecting fixed point and points . The length of these line segments gives us the distance between the fixed point and each of the three points on line .

The distance between fixed point and line is thus

.

We have an optimization problem: we would like to find the value of parameter which minimizes function . This minimizer is (see appendix A)

.

Evaluating function at we obtain , the point on line that is closest to fixed point

,

where

is a symmetric, idempotent () projection matrix, and is the orthogonal projection of vector onto vector . It can be shown that vectors  and are orthogonal (see appendix B).

For example, let us consider the case: we have two lines in , and , where lies on plane . Pictorially, we have

We would like to find the point on line that is closest to fixed point . Projecting vector orthogonally onto vector , we obtain

The image above illustrates rather nicely that vectors  and are orthogonal (see appendix B). The distance between fixed point and line is

,

where . Please note that plane serves no purpose other than to make it easier to visualize in 3-d (plane gives an impression of depth). In other words, plane has some “artistic value”, but no “mathematical value”.

__________

Distance between two lines

Let us now vary both and . As we vary and , we travel along lines and , respectively. Since both parameters are free, the distance between free points and is

,

which is a function of both and . The distance between lines and is thus

.

The original problem of computing the distance between two lines has thus been cast as an optimization problem: we would like to find the pair which minimizes . Once we have found the pair of minimizers , the distance between and is

.

The pair is not necessarily unique. For example, if the lines are parallel to each other, then there are infinitely many pairs of minimizers .

Trying to find the pair which minimizes is a one-stage 2-dimensional optimization problem. However, we can solve this optimization problem in two stages, where at each stage we must solve a 1-dimensional optimization problem

.

Note that

is the distance between point and line , and thus

.

We already know how to compute the distance between a point on line and line : given a value of , the choice of which minimizes is , where function is defined as (see appendix A)

,

and the point on line which is closest to is

.

Therefore, for a given value of , we have that the distance between and line is

.

Note that is a function of only, as we have already optimized function for . Finally, the distance between lines and can be found by solving a 1-dimensional optimization problem in

.

Once we have found the which minimizes the distance , the distance between lines and is simply

.

Note that:

• : lines and do intersect.

• : lines and do not intersect.

The value of can be written in terms of points and direction vectors , as I will show on some future posts.

__________

The shortest distance game

It might seem a bit obscure that the one-stage 2-dimensional optimization problem

is equivalent to a two-stage optimization problem where at each stage one must solve a 1-dimensional optimization problem

.

Personally, I like to look at this from a game-theoretic viewpoint. Let us play a sequential game! We shall call the shortest distance game. I play first, then you play:

1. I choose a value for , I tell you what I chose.
2. You then choose such that is minimized.

Note that you are required to play optimally, i.e., to minimize distance. Hence you have no freedom in the choice of : you choose as a response to my choice of . Your choice is now coupled to mine, and thus, we are left with only one degree of freedom, the choice of . I now choose the value of which minimizes

.

Fortunately, I can use calculus to find the minimizer . Trying all values in for parameter would be rather tedious ;-)

__________

Appendix A

Note that the value of parameter which minimizes the distance is the same that minimizes the squared distance . Finding the minimizer of the squared norm is much easier, thus we introduce a cost function defined as

.

Differentiating with respect to , we obtain

The first derivative of must vanish at extrema. From the geometry of the problem, we know that there are no maxima (note that the cost function is unbounded above). Therefore we can find minima by checking the first derivative alone, without having to check the second derivative. We have

,

and since , the minimizer is

.

Note that this minimizer depends on the value of fixed parameter .

__________

Appendix B

We would like to show that vectors and are orthogonal. Recall that , and thus

.

Multiplying both sides on the left by , we obtain

and since (recall that ), then we finally have

,

which means that vectors and are orthogonal.

__________

Related:

• About Lines and Distance of a Point to a Line (by Dan Sunday)

• Distance between Lines and Segments with their Closest Point of Approach (by Dan Sunday)