In his Oct. 2, 2019 article Andrew Ng wrote: “Despite progress from typewriters to text editors, why is writing still hard to do? Because text editors don’t address the most difficult part: thinking through what you want to say.

Programming tools have the same limitation. I’m glad to be coding in Python rather than Fortran. But as Brooks points out, most advances in programming tools have not reduced the essential complexity of software engineering. This complexity lies in designing a program and specifying how it should solve a given problem, rather than in expressing that design in a programming language.” Read more

“Deep learning is revolutionary because it reduces the essential complexity of building, say, a computer vision system. Instead of writing esoteric, multi-step software pipelines comprising feature extractors, geometric transformations, and so on, we get data and train a neural network. Deep learning hasn’t just made it easier to express a given design; it has completely changed what we design.”

“I don’t know what will be the key ideas for reducing this essential complexity, but I suspect they will include software reuse, ML model reuse (such as libraries of pretrained models) and tools not just for code versioning and reuse (like github) but also for data versioning and reuse. Breakthroughs in unsupervised and other forms of learning could also play a huge role.”