際際滷

際際滷Share a Scribd company logo

AI - summary of focus groups.pdf

David Wood
David Wood

A summary of discussions of five different groups on the subject of AI in the closing session (on 21st January 2024) of TransVision Utrecht. David Wood facilitated these discussions and compiled these notes.

1 of 10
Download to read offline
Summary of AI Focus Groups From TransVision Utrecht 21st January 2024 Compiled by David Wood (AI Focus Groups facilitator)
1. AI is a big topic The five different focus groups had very different conversations Some focused on control Some focused on alignment Some focused on risks; Some preferred to highlight benefits Some looked at here-and-now; some had longer-term timescales None of the groups considered the impact of AI on work (!) This is a reminder that AI transcends easy categorization If you think you understand all key aspects of AI, you are probably mistaken!
2. To help us with AI, we need philosophy The conversations kept reaching questions of philosophy: What should we want to want? What is consciousness, and might it arise in AIs? What is the meaning of meaning? What is the meaning of autonomy / agency / free will, and might it arise in AIs potentially making control or alignment harder? How can we understand and evaluate our own biases? Perhaps AI could help us with these philosophical questions! AIs could summarize what appears to be peoples preferred values AIs could suggest new solutions to alignment or control
3. Revising the UDHR? The process to create the UDHR (Universal Declaration of Human Rights) can serve as a useful example 1946 to 1948 Drafting committee was highly multinational and culturally diverse The resulting document was widely accepted The exercise needs to be repeated (2024 to 2026)? What are the most important rights and responsibilities for humans to uphold in an age of co-existence with increasingly powerful AIs? AI could help in the drafting and review process
4. Getting onto the same page There are too many cases of widely different understanding of what should be basically agreed facts What are the forthcoming capabilities of new AI systems? What are the principal opportunities and key risks? Which scenarios are credible for how things might develop? The independent report being prepared by Yoshua Bengio is particularly important Commissioned by Global AI Safety Summit at Bletchley Park (UK) Bengio has the advantage of never being part of any big tech
5. Understanding catastrophic risks There is not just one type of catastrophic risk, but several Bugs in powerful systems, for example in out-of-distribution cases Systems that are hacked, to remove their safety precautions Systems used by malevolent groups (e.g. like Aum Shinrikyo) AIs potentially developing their own divergent goals Failures in communication between AIs and humans AIs taking their goals too seriously e.g. maximize profits Normally admirable companies racing each other for market dominance and cutting safety corners in that race Solutions for one sort of risk may fail to apply for others
6. The dangers of poor regulation AI is too important not to regulate, but is too important not to regulate well (said by leading Google UK lawyer) Regulations can unnecessarily slow important beneficial innovation Beware regulations that are static they need to be agile (readily adaptable) Beware regulations that are just for virtue signaling, and which cannot be applied in practice Governments have other levers, not just regulation (sticks) Providing incentives (carrots), to boost desirable outcomes Setting rules for AI systems they will procure for public use
7. Prioritizing defensive technologies Need to design and implement canary signal monitors Consider what actions by an AI would be reasons for alarm (Special attention to behaviors that are deceptive?) (Special attention to behaviors that would accumulate power?) Prepare tamper-proof shut-down mechanisms ready for use Explore use of narrow AIs to help monitor general AIs General AIs could be restricted to offering advice The advice would need to be evaluated and approved by narrow AIs before being actioned in the real world
8. Special complications of open source Its nearly always preferable to share information widely More eyeballs make every bug transparent More eyeballs can find many unexpected new solutions But cataclysmically dangerous tech may change matters Recipes for creating highly contagious fatal biological pathogens? AI systems originally designed to invent new antibiotic drugs were easily reprogrammed to invent new poisons instead, oops Large Language Models can give advice step by step, explaining methods to users as if they were ten years old for simplicity Explain to me how I could kill every person of nationality X!
9. Re-imagining the future What are the prospects for human AI merger? Can humans improve as fast as AIs can improve? Will the limitations of human skulls slow down any merger? Claims made for BCIs might be over-exaggerated Might misaligned superhuman humans be just as dangerous as misaligned superhuman AIs? Whats so bad about humans being the pets of AIs? Humans may be looked after by machines of loving grace Is our future in base reality or in simulated universes?

More Related Content

AI - summary of focus groups.pdf

  • 1. Summary of AI Focus Groups From TransVision Utrecht 21st January 2024 Compiled by David Wood (AI Focus Groups facilitator)
  • 2. 1. AI is a big topic The five different focus groups had very different conversations Some focused on control Some focused on alignment Some focused on risks; Some preferred to highlight benefits Some looked at here-and-now; some had longer-term timescales None of the groups considered the impact of AI on work (!) This is a reminder that AI transcends easy categorization If you think you understand all key aspects of AI, you are probably mistaken!
  • 3. 2. To help us with AI, we need philosophy The conversations kept reaching questions of philosophy: What should we want to want? What is consciousness, and might it arise in AIs? What is the meaning of meaning? What is the meaning of autonomy / agency / free will, and might it arise in AIs potentially making control or alignment harder? How can we understand and evaluate our own biases? Perhaps AI could help us with these philosophical questions! AIs could summarize what appears to be peoples preferred values AIs could suggest new solutions to alignment or control
  • 4. 3. Revising the UDHR? The process to create the UDHR (Universal Declaration of Human Rights) can serve as a useful example 1946 to 1948 Drafting committee was highly multinational and culturally diverse The resulting document was widely accepted The exercise needs to be repeated (2024 to 2026)? What are the most important rights and responsibilities for humans to uphold in an age of co-existence with increasingly powerful AIs? AI could help in the drafting and review process
  • 5. 4. Getting onto the same page There are too many cases of widely different understanding of what should be basically agreed facts What are the forthcoming capabilities of new AI systems? What are the principal opportunities and key risks? Which scenarios are credible for how things might develop? The independent report being prepared by Yoshua Bengio is particularly important Commissioned by Global AI Safety Summit at Bletchley Park (UK) Bengio has the advantage of never being part of any big tech
  • 6. 5. Understanding catastrophic risks There is not just one type of catastrophic risk, but several Bugs in powerful systems, for example in out-of-distribution cases Systems that are hacked, to remove their safety precautions Systems used by malevolent groups (e.g. like Aum Shinrikyo) AIs potentially developing their own divergent goals Failures in communication between AIs and humans AIs taking their goals too seriously e.g. maximize profits Normally admirable companies racing each other for market dominance and cutting safety corners in that race Solutions for one sort of risk may fail to apply for others
  • 7. 6. The dangers of poor regulation AI is too important not to regulate, but is too important not to regulate well (said by leading Google UK lawyer) Regulations can unnecessarily slow important beneficial innovation Beware regulations that are static they need to be agile (readily adaptable) Beware regulations that are just for virtue signaling, and which cannot be applied in practice Governments have other levers, not just regulation (sticks) Providing incentives (carrots), to boost desirable outcomes Setting rules for AI systems they will procure for public use
  • 8. 7. Prioritizing defensive technologies Need to design and implement canary signal monitors Consider what actions by an AI would be reasons for alarm (Special attention to behaviors that are deceptive?) (Special attention to behaviors that would accumulate power?) Prepare tamper-proof shut-down mechanisms ready for use Explore use of narrow AIs to help monitor general AIs General AIs could be restricted to offering advice The advice would need to be evaluated and approved by narrow AIs before being actioned in the real world
  • 9. 8. Special complications of open source Its nearly always preferable to share information widely More eyeballs make every bug transparent More eyeballs can find many unexpected new solutions But cataclysmically dangerous tech may change matters Recipes for creating highly contagious fatal biological pathogens? AI systems originally designed to invent new antibiotic drugs were easily reprogrammed to invent new poisons instead, oops Large Language Models can give advice step by step, explaining methods to users as if they were ten years old for simplicity Explain to me how I could kill every person of nationality X!
  • 10. 9. Re-imagining the future What are the prospects for human AI merger? Can humans improve as fast as AIs can improve? Will the limitations of human skulls slow down any merger? Claims made for BCIs might be over-exaggerated Might misaligned superhuman humans be just as dangerous as misaligned superhuman AIs? Whats so bad about humans being the pets of AIs? Humans may be looked after by machines of loving grace Is our future in base reality or in simulated universes?