SPINLAB @ MSU

A lecture by Claude Shannon on Creative Thinking is buried deep inside of his Miscellaneous Writings edited by N. J. A. Sloane & Aaron D. Wyner in 1990. It has been rediscovered in some sense by the authors Jimmy Soni & Rob Goodman who have just recently published a biography of Shannon. One of the authors, Soni, describes 12 things that they learned about Shannon while researching their book. Below are my notes from his Creative Thinking lecture, which is transcribed here by Yousof Naderi, as well as Bob Gallagher’s talk on Shannon (see above with the corresponding slides below) as a part of the 2015-2016 Boole/Shannon Celebration:

My personal notes from the article and talk above:

• A capable researcher must have the following three traits:
  • training and experience
    • learn a lot of facts – achieving this is often boring & requires self-discipline 
    • have deep knowledge of the background material (i.e. know what is already known)
    • compare this to step 1 of “A Technique for Producing Ideas” by James Webb Young summarized by James Clear:
      • step 1: “gathering material”
      • step 2: “intensely working over the material in your mind”
      • step 3: “stepping away from the problem”
      • step 4: “allowing the idea to come back to you naturally”
      • step 5: “testing your idea in the real world and adjusting it based on feedback”
      • side note: compare steps 3 and 4 above to Hadamard’s book The Mathematician’s Mind – “The roots of creativity for Hadamard lie not in consciousness, but in the long unconscious work of incubation, and in the unconscious aesthetic selection of ideas that thereby pass into consciousness.”
  • intelligence
    • hard to define, somewhat hereditary
    • “There are some people if you shoot one idea into the brain, you will get a half an idea out. There are other people who are beyond this point at which they produce two ideas for each idea sent in.”
  • intellectual motivation, specifically:
    • an inner drive to formulate questions and answers
    • a curiosity about fundamental characteristics – what makes things tick (big picture and not details)
    • a need to understand in multiple ways, probe all sides of the problem;  compare to ‘t Hooft‘s How to become a GOOD Theoretical Physicist (older version), The World from Another Point of View, and this Nobel Lecture for the 1965 Physics Prize.
    • an appreciation of the cleverness of an idea; getting deep satisfaction & enjoyment from understanding something new (the “A-ha” moment)
      • compare to Fun to Imagine & The Pleasure of Finding Things Out
      • “[A researcher] should probably have an extremely strong drive to want to find out the answers, so strong a drive that [she] doesn’t care whether it is 5 o’clock – [she] is willing to work all night to find out the answers and all weekend if necessary.”
    • a sense of “constructive dissatisfaction” which can be defined to be a deep desire to iteratively:
      • develop an idea just a little further
      • make an argument neater
      • improve things just a little more
• Shannon’s Tricks (or Major Principles of Research) from his lecture
  1. Simplification: get rid of enough detail (including practical aspects) for intuitive understanding; simplify first and then build it back up
  2. Similarity to a known problem (experience helps); compare to Hamming
  3. Reformulate (avoid getting in a rut); when stuck, stop and look for a completely different way to look at the problem
     • Change the words.
     • Change the viewpoint.
     • Describe the problem to someone “green” (outside of the field) and ask them to repeat it back to you in their own words
  4. Generalize (more than opposite of simplify); strip away one of the simplifying assumptions and solve the problem again
  5. Structural analysis (break problem into pieces); solve a microproblem first and then build up – divide & conquer (compare to Pólya’s How to Solve It)
  6. Inversion (work back from desired result); work backwards from what the answer ought to look like
• “Other tricks that Shannon often used” from Bob Gallagher’s observations:
  1. Be interested in several interesting problems at all times. Work on the most interesting one.
     • “One of the most difficult things to learn how to do is knowing when to give up on a problem. However, if you are working on several problems, then you will slowly and incrementally keep on making progress on the good ones which you will eventually solve. On the other hand, you won’t make much progress on the bad ones and eventually they will recede in your mind and will effectively be dropped. You can not reason your way out of a bad problem because the knowledge required to determine the “badness” of a problem is also the knowledge that you need to properly define and then solve the problem in the first place.“
  2. Look for contradictions as well as proofs.
     • Be critical of the idea – when you think you understand something, try to blast as many holes as possible into it.
  3. Study what is happening in multiple fields, but don’t work on what many others are working on. Compare to EWD 637:
     • “If you work on a problem that many others are working in, then you’ll have to work fast if you want to be successful. The likelihood that you’ll be successful will also be less.”
  4. Ask conceptual questions about everyday things.
  5. Don’t write papers unless you really want to share something fascinating.
  6. Don’t assume your readers know everything you do. Spoon feeding is not a bad idea.
• Shannon was a big nerd and not a athlete in high school, graduated when he was 16 and then from college at the University of Michigan at 20.
• Worked with Vannevar Bush @ MIT for his Master’s thesis, which laid the foundation for digital electronics
• Switched to the mathematical theory of genetics for his PhD thesis, lost interest in it, never published it, several results were rediscovered by others laters, not a big deal since he was already famous
• An example of how he defined one of the problems that he worked on:

  The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design. – C. E. Shannon, 1948

• His closest collaborator was his wife Betty Moore Shannon.
• He built mathematical (and physical) models to help understand these problems, but his focus was on the underlying problem (the architecture), not in mathematics per se nor in problem details. Shannon was a creator of models — his genius lay in determining the core of the problem, removing details that could be reinserted later. Compare to Modeling Framework for Experimental Physics.

Gallagher’s talk is worth comparing and contrasting with Hamming’s Creativity lecture. Remarkably, they were both driven to think deeply about the creative process because of their relationship to Shannon as a friend & colleague as well as their appreciation of his deep & influential body of scientific work. One hole that Gallagher plugs in the process that Hamming describes is how to drop bad problems while at the same time not dropping a good problem too early, see bold text above.

-JTS