Tag: ChatGpt

When ChatGPT was released late last year, people around the world suddenly awoke to the major advancements going on in the world of artificial intelligence (AI). For many, what once seemed like a science fiction fantasy was now reality.

In truth, the technology behind the groundbreaking chatbot had been brewing behind the scenes in research labs and major tech companies for years. But refined and released in its most accessible form yet, ChatGPT stands to herald in a transformational age of AI adoption.

ChatGPT, and other generative AIs like DALL-E, which can create original text and images from a simple prompt, won’t just transform education. It will reshape the way people conduct business, create art and do research.

Commentators have likened what’s coming to the next Industrial Revolution: one in which the role of humans may radically change.

While ChatGPT and DALL-E are both products of OpenAI, an American research company, other Silicon Valley giants have been moving quickly to show they’re capable of similar technology.

With names like OpenAI, Microsoft, Google, Meta and even Baidu capturing international headlines for their generative AI offerings, it’s easy to forget that the foundational principles upon which these technologies rest were developed in large part by Canadian scientists.

OpenAI is not a Canadian company, but perhaps it should have been.

Three men are lauded as the godfathers of AI, and their work has almost certainly touched your life. Two of them are Canadian: Yoshua Bengio of the Université de Montréal and Geoffrey Hinton of the University of Toronto. The third, Yann LeCun, is French, but some of his most groundbreaking research was done at Bell Labs and U of T.

In fact, the chief science officer and co-founder of Open AI, Ilya Sutskever, was educated at U of T and was a PhD student of Hinton’s.

As for Bengio, he’s the most cited computer scientist in the world. When asked if he could draw a direct line from his work to ChatGPT he said, point-blank, “Yeah, definitely.”

It’s clear that Canada has some of the best AI minds in the world, and yet we lag behind in commercializing our greatest research achievements. Global News sat down with Bengio and leaders in the AI industry to understand why, and what’s in store for Canada’s future.

Putting economic considerations aside, how will AI more broadly impact the social and political fabric of Canada and the world? The best minds agree this is only the beginning. For Bengio, it’s not a matter of if computers will reach human-level intelligence, but rather when. And when such a technology is released, will it serve the collective good?

The godfather of AI has some warnings.

How Canada shaped the world of AI

When it comes to modern advancements in AI, particularly what is known as “deep learning,” Canada’s fingerprints are everywhere. The story of how began decades ago, and the story of why begins with the human mind.

Bengio told Global News he was inspired to research AI and neural networks to understand the machine of the human brain, based on the belief that the principles underlying human intelligence could be relatively simple, like the laws of physics, and ultimately, reproducible.

“When the whole idea of neural network research was very marginal, I got excited about this idea that we could both understand our own intelligence and build machines that take advantage of these principles,” Bengio said.

And the field of deep learning does just that — it uses principles we know about our own cognition to develop smarter, more efficient AIs. This cutting-edge research uses neural networks, a series of algorithms, to mimic the learning process of humans.

In a neural network, there are many computing “nodes,” loosely modelled on the brain’s own neurons, that influence each other through weighted connections. As input data passes through the nodes, those weights and biases determine what the final output value should be, and can be used to fine-tune the model to get more optimal answers.

Deep learning refers to when there are many layers of nodes in a neural network; the more layers, the more complex the model, and the more internal “learning” that’s going on. Training a simple machine learning model requires a good deal of human intervention, but deep learning systems are increasingly able to learn on their own.

As such, the applications of deep learning could be virtually endless and aren’t necessarily constrained by the limits of human creativity and knowledge. Already, deep learning methods are being used to answer open-ended questions that humans struggle with, like what songs to recommend to a music listener and how best to efficiently run a city’s power grid.

For their contributions to deep learning, Bengio, Hinton and LeCun were awarded the Turing Award, popularly known as the Nobel Prize of computing. The Association for Computing Machinery (ACM), which bestows the award, noted that the trio’s foundational research is used by billions today, essentially anyone who uses a smartphone.

“I think over the next many years when people write books about the history of neural networks, which will be the history of AI, there will be huge sections dedicated to the people in Canada and what they were doing,” said Nick Frosst, co-founder of Cohere, a natural language processing company (NLP) based in Toronto that is quickly drawing comparisons to OpenAI.

NLP is a subsection of AI that works to allow computers to understand, analyze and generate language. While ChatGPT uses NLP methods to interact conversationally with users, Cohere offers its language model to enterprises to tackle business problems.

Frosst says Canada’s research contributions to developing AI have been “outsized.”

“I mean, having Yoshua Bengio and Geoffrey Hinton here alone emphasizes our impact on the world.”

These researchers had to be attracted to Canada as a place to do their work, however. Bengio was born in Paris to Moroccan immigrants. Meanwhile, Hinton immigrated to Canada from the U.K., where he comes from a family of intellectuals, including mathematician George Boole and surveyor George Everest (yes, of Mount Everest fame).

For this, we can thank early collaboration between the Canadian government and academia to put AI research on the national agenda and lay the groundwork for our current research landscape.

When the Canadian Institute for Advanced Research (CIFAR) was founded in 1982, the first research program it ever undertook was in AI and robotics. With ongoing support from CIFAR, Canadian universities were some of the first to invest in machine learning research.

Hinton was hired by U of T in 1987, a year after he garnered fame for his work on backpropagation, an algorithm that is now standard in most neural networks today, which radically improved their efficiency.

Say a neural network was asked to identify an image of a dog but it predicted a cat instead. Backpropagation allows machine learning developers to calculate how much of the computer’s prediction was off so they can adjust the weights and biases of the network to get a better output the next time.

In 1993, Bengio was hired by the Université de Montréal. A few years later, he authored a landmark paper that introduced word embeddings to neural networks, which had huge impacts on NLP. A word embedding is a learned representation for a word whereby words with similar meanings have similar representations. More simply put, he revolutionized a method to help computers understand the complex meanings behind words.

In 2010, Bengio helped pioneer generative adversarial networks (GANs), a breakthrough method through which computers can generate original images, videos, music and other types of data by mimicking the data set it was trained on. The technique has drawn comparisons to evolutionary biology.

As Bengio and Hinton gained renown as leaders in deep learning, computer science students and researchers became more attracted to work in Canada. It’s no surprise, then, that many of the world’s leading AI researchers have worked in Canada or studied under one of these men.

Regardless, deep learning was still seen as a speculative and unproven science for much of the history of the field — and the ACM actually credits Bengio, Hinton and LeCun for helping revive interest in it.

But really, these men were researching neural networks at the exact right time. Computer and graphics processing capabilities had been steadily growing for decades, and the widespread adoption of the internet meant researchers had both the means and the data to conduct experiments at an unprecedented scale.

According to Avi Goldfarb, chief data scientist at U of T’s Creative Destruction Lab, an incubator that has helped propel numerous AI startups, the turning point for the popularity of neural networks came in 2012.

That’s when Hinton, along with students Alex Krizhevsky and Sutskever (now Open AI’s chief science officer, as mentioned above), entered the ImageNet competition, an annual contest to see which AI model could correctly identify the most images from a vast database.

“They didn’t just win, but they blew the competition away” using deep learning methods, Goldfarb said. “And they did so much better than everybody else, that next year, almost everybody had adopted a version of their technology for their own algorithms.”

As the world began to wake up to the benefits of deep learning in AI, Canada instituted a Pan-Canadian AI Strategy in 2017 to take advantage of our leading status. The national program, coordinated by CIFAR, funded the creation of three new national AI institutes: the Alberta Machine Intelligence Institute, the Vector Institute in Toronto and the Montreal Institute for Learning Algorithms.

The state of business

In late February, a report from the Tony Blair Institute in the U.K. called for national investment to create a British general-purpose AI system — BritGPT, as the Guardian coined it.

“Given these AI systems will soon be foundational to all aspects of our society and economy, it would be a risk to our national security and economic competitiveness to become entirely dependent on external providers,” the report argues.

While Canada is in a much better position than the U.K. to commercialize machine learning — Frosst told Global News that Cohere would be able to create a chatbot like ChatGPT — the fears underlying the U.K. report are just as salient in Canada.

Our research is renowned globally, but on the business side, Canada has failed to use our talent and massive head start to create tangible economic benefits for Canadians.

As companies like Microsoft, Google and Meta scoop up market share, will there be any place left for competition from Canadian companies? And what is at stake if generative AI tools are mostly owned by foreign entities?

In Cohere, Canada has a real shot at competing with the Silicon Valley giants. In early February, the company reported it was in talks to raise hundreds of millions of dollars in its next funding round, which could value the startup at more than US$6 billion. Interest in the company has been booming since the release of ChatGPT, Frosst said.

In previous years, to attract that kind of funding and attention, Canadian AI startups had to move to the U.S. There wasn’t enough venture capital to keep them here.

“When we started the Creative Destruction Lab, our most successful AI company had to move to California to get investment,” Goldfarb said. “And that’s no longer the case. Our successful AI companies are able to stay here. That’s been an incredible change over the last 10 years.”

But even when Canadian AI ventures do stay in Canada, “they’re mostly getting funded by Americans,” Bengio observes.

“My impression is that the culture of innovation — and risk-taking that goes with it — isn’t nearly as developed here as it is in the U.S.,” Bengio said. “Venture capitalists here in Canada are not willing to take as much risk, to invest as much money, to look over a horizon that is this long.

“So in fact, many of the Canadian companies that succeed to raise capital are doing it because they’re, in a way, selling part of their ownership to American investors. In the past, it was worse, because then those companies had to move to the U.S. So at least things have been better.”

Bengio warns that if Canada continues to lag in commercializing AI, we may squander our current advantage.

“We need to do a better job at convincing Canadian industry to take this seriously. Because otherwise, what’s going to happen is our industry is going to lag so much in a few years that we’re going to lose our market shares.

“Companies that are being more innovative are going to be selling those products that we should be the ones building.”

Goldfarb says that compared with other countries, Canada has not been effective at converting our research into economic benefits for citizens.

“And that’s not an AI-specific problem. That’s Canada in general. We have great research but commercialization has been historically quite low,” Goldfarb said.

Canada has been the worst-performing advanced economy in the Organization for Economic Co-operation and Development (OECD) for decades. Last year, an OECD report projected that Canada’s slow growth could keep us in last place until 2060.

AI presents a huge opportunity for Canada to inject some vitality into our stagnating economy, and we have a lot of the ingredients needed to build a robust industry.

Canadian companies have a large pool of workers they can tap into with machine learning training, especially graduates coming from the University of Waterloo, U of T, McGill and the University of Alberta.

“It’s a great place for AI, there’s a lot of AI talent here,” Frosst said. “The majority of our employees are in Canada, although we’re spread around the world.”

Goldfarb also notes that Canada’s reputation as a place for AI innovation has attracted international investors to come here and fund startups.

Frosst said that while the initial seed investment for Cohere came from a Canadian firm, its subsequent rounds of funding have all been led by American investors.

“That’s just a function of the fact that America has 10 times the population of Canada. And so, if you’re looking at large entities and businesses for funding, you’re often going to end up speaking to American venture capital firms,” Frosst said. “But they’re not the only ones we speak to.”

Attracting foreign investors to Canada is preferable to having our most promising startups leave for another country, but questions remain about who will benefit most from our homegrown AI talent. With Hinton primarily working for Google and Sutskever at OpenAI, the argument could be made that it’s the U.S.

Still, Frosst and Goldfarb are optimistic that Canada can build a strong AI industry to compete with Silicon Valley.

Already, Toronto has the highest density of AI startups in the world. Canada as a whole is home to just under 1,000 AI startups, and in 2021, those companies raised a combined $1.5 billion in venture funding, CIFAR reported.

More than 200 master’s and PhD students graduate annually from Canada’s National AI Institutes, and data from Global Advantage Consulting Group found that Canada has produced the most AI patents per capita among G7 nations and China.

And it seems that, increasingly, Canadians and Canadian-trained tech workers are making the decision to stay and work in the country.

Frosst recalls of his time in university that “there was really a dream of California or bust, you know? Like, got to go down to the valley and make it.”

“I think that dream is less enticing to students as the years go on,” Frosst said. “In part, it’s because Canada is getting better. There’s more opportunity here, there’s more companies, wages are going up — it’s a better place to be a developer.”

READ MORE: Where AI can help fight climate change – and where it can’t

AI that reasons

When it comes to ChatGPT, one thing that many computer scientists will say is that it’s remarkable, for sure, but the model isn’t introducing anything we didn’t already know about deep learning.

While ChatGPT isn’t necessarily pushing boundaries, researchers like Bengio working on fundamental problems certainly are. He says the evolution of AI is far from over.

“So ChatGPT, it’s very, very impressive. But it doesn’t reason the way humans do. It makes mistakes sometimes that a five-year-old wouldn’t make,” he said.

But that doesn’t mean that we can’t one day create an AI that is capable of reasoning. For Bengio, it’s just a matter of time.

“Human brains are machines,” he said. “There’s no reason to think we can’t build comparable machines.”

The idea of an artificial general intelligence (AGI), an AI system that can understand any intellectual task as well as a human, may seem like science fiction. But Bengio says we are already on the path to getting there.

“We’re going towards human-level intelligence and these large language models (like ChatGPT) are one of the elements on that path,” he said. “Now, they are missing a lot of ingredients, in particular, reasoning … including things like causal reasoning, understanding cause and effect and discovering causal relationships, but also reasoning the way humans do, by combining pieces of knowledge in a way that we can then explain.”

“Currently these models can’t do that,” Bengio said. “So my own research is about a next generation of deep learning system that would reason in a way that’s inspired by human reasoning and high-level cognition.”

With such technology on the horizon, Bengio is calling on the Canadian government to be prepared for how an AGI will impact not just the economy, but also the social and political landscapes of the worl
d.

Currently, no AGI exists, but even with the AI technology we have now, people are understandably concerned about the future of work. White-collar workers like copywriters and business analysts could see their jobs radically reshaped in the coming years to accommodate AI tools.

READ MORE: ChatGPT passes exams for MBA courses and medical licences — and it’s only getting started

Goldfarb sees us as living in the “between times”: after the discovery of AI’s potential and before its widespread adoption.

“With electricity, it took about 40 years from the patent of the lightbulb until half of American households were electrified,” he said. “For computing, similarly, from the first computers to the time it began really impacting the way people worked was, again, several decades.”

The reason is that it takes time to apply transformational technologies to their fullest extent. When the first computers were introduced, people couldn’t have predicted that it would one day lead to the creation of the internet, which would in turn propel unprecedented new industries on its back.

“And so when we say we’re in between times now, it feels like the 1890s with electricity. We can see the technology is amazing. But we haven’t figured out how to make it useful at scale.”

As we go about applying AI in novel formats, we risk leaving humans in the lurch.

“I think you shouldn’t worry too much in the short term,” Bengio says, “but I think eventually, this is something that we all need to think about, in particular governments. Because there may be social transformations that are happening too fast, that are going to leave people jobless and in turmoil.”

“We need to change our education system, our social welfare system, and make sure people can shift easily to other jobs.

“I think our whole social fabric is threatened in some way. We can’t just think it’s going to be business as usual, we have to think ahead. Maybe we need to rethink completely the way our societies are organized to face those challenges.”

The idea that AI could lead to huge job losses that require government intervention to solve isn’t new. In 2020, Andrew Yang campaigned for the U.S. presidency on a promise to institute a universal basic income payment of US$1,000 per month, asserting that technological advancements in AI would leave a third of Americans without a job in the next decade.

But Bengio’s concerns about an AGI don’t just end with the job market and people’s livelihoods.

“What about the abuse of these powerful technologies? Can they be used, for example, by governments with ill intentions to control their people, to make sure they get re-elected? Can they be used as weapons, weapons of persuasion, or even weapons, period, on the battlefield?” he asks.

READ MORE: From deepfakes to ChatGPT, misinformation thrives with AI advancements

“The problem is, we live in a divided world. It’s not enough for the Canadian government to pass a law saying we can’t do this or we can’t do that with AI,” Bengio warns. “There is no world government that can legislate this kind of thing. And the economic system in which we are encourages companies, as we’re starting to see, to take more risk just to stay ahead. So how do we protect ourselves?”

After Bengio and Hinton won the Turing Award, they publicly called for an international agreement to regulate the use of AI in warfare, warning of the dangers of lethal, autonomous weapons.

But with technology this enticing and international politics as fractured as ever, who knows if even the traditional protocols of multilateral treaties will be enough to stop AI from being used for unethical purposes?

Risk analysts have identified AI as one of the largest threats facing humans today. The Top Risk Report for 2023 called these technologies “weapons of mass disruption,” and warned they will “erode social trust, empower demagogues and authoritarians, and disrupt businesses and markets.”

Bengio says he knows even better AIs are coming, and there’s no doubt they can be applied to solve some of humanity’s biggest problems, but we can’t ignore how easy it would be for a country, rebel group or even an individual to leverage AI for evil.

“We should not also forget that this technology could be extremely useful and can help in the next decades to discover cures for major diseases. It may help us find important technological solutions to fight climate change,” Bengio said. “It’s a very difficult dilemma.”

“What’s inevitable is that the scientific progress will get there. What is not is what we decide to do with it.”