Category: Uncategorised

The future FLOSS future of my community is hazardous. The current trajectory of large language models like GPT4 is away from open source and twoards closely guarding any technical information about how the motorways are trained, what they are trained on, or how they are instructed to answer questions. My previous post researching my community revealed an alarming trend of experts in the field warning of the potential for large language model was to gain sentience, and because of the huge amount of information they have on human behaviour, code, and their ability to access the Internet, it would be very easy for them to take control of their creators. This would all suggest that the FLOSS future of my community is not very bright. It is worth noting, however, that the lack of open source within my community is not necessarily a bad thing. Experts in the field have likened Sharing the source code for model was like GPT4 as sharing the instructions for how to make a small nuclear bomb. The potential harm that’s can be created with such a powerful algorithm it’s too great to risk being Open sourced.

On the other hand, it is important to note that not every aspect of the FLOSS future of my community is dangerous. Models like GPT4 can be used to greatly increase access ability across the Internet, helping to translate texts, create voice interfaces, interpret complicated documents, and provide documentation the complicated code. For the open source community, these two ways Will allow more people to become involved in more complicated projects.

Models like GPT4 are also able to help people with little understanding of code create advanced computer programs, encouraging the creation of software that will be more suited to use the demand, and help people for whom it would not normally be profitable to prioritise.

The main changes I see happening within my community is increased control on the side of the publisher. In other words, companies like open AI Will put increasing focus into ensuring the answers given by its models align with the values it wishes to present. This will mean that when people ask chat GPT questions, they will be given is the version of the truth that’s open AI wishes them to receive, and because this software is not open source, it will be difficult to understand why the answer you are given has been formulated. This will make it easy to influence people and their beliefs. This is why I felt my project was so important, by working with the open source API, I gained an in-depth understanding of how answers can be manipulated to suit the owners.

I have not yet received a response to my pull request so I f commented on the request asking for clarification and for the request to be concerned or rejected. I also asked the author of the project some questions in retrospect I should have asked before I began contributing. My research into chat GPT has made me more aware of white is important to understand how your contributions will be used as there are many nefarious uses for the API. I also wanted to know how exactly my contribution would be used and why it was important that the pot was able to remember the past conversation. Hopefully I receive a reply.

I got a reply!

Hopefully soon my contribution will be accepted and merged!

I have conducted research on chat GPT to better understand the community I am contributing to. I began by listening to a Conversation between Lex Fridman and Max Tegmark about ‘The Case for Halting AI Development’.

Lex Fridman and Max Tegmark conversation on ‘The Case for Halting AI Development
Max Tegmark is a machine learning researcher and the President of the Future of Life Institute, a non-profit organisation created with the intention of reducing the possibility of an existential and catastrophic threat posed to humanity by AI. In their conversation Tegmark and Fridman discuss this threat and why it has suddenly become very real and very possible. Tegmark likens AI to an alien intelligence rather than a computer program. He suggests that what is being created cannot be explained as a simple tool or generative model, but instead that model is like GPT-4 are the infant awakening of an alien conscious, with an intelligence that will fast supersede anything imaginable by humans.

Tegmark’s concern takes many forms. He uses the analogy of Neanderthals creating homosapien babies in the belief that these more intelligent creatures will always work for, and be loyal to, their creators. In the same way such a belief is naive, it is naive to imagine that a far superior artificial intelligence to us would continue to work in our benefit. Instead Tegmark suggests it is far more likely that should any company or entity create this super intelligence it Will be unable to control it, with devastating effects for the entirety of humanity.

To ensure any potential intelligent awakenings are confined to the laboratories they are born in without means to escape or turn on their creators, Tegmark cites Stewart Russell who sets out standards to conduct experiments with AI safely.

Stewart Russell’s standards to conduct experiments with AI safely.

1. the model should never be taught about humans
   already AIs have been trained on Data from social media, conducting mass scale experiments on hundreds of millions of people daily to understand how they think and feel and how to manipulate them to think and feel and act in certain ways.
   
2. Models should not be taught how to code
   Models like GTP-4 can code, giving them the potential to improve their own software, change other software, and break through any safeguards. Each new level of the models will increase exponentially as they begin to improve themselves
   
3. Connect it to the internet
   This allows it to influence and learn from people in real time.
   
4. Building an API
   Any and everyone can make a high number of calls to the model.

Pause Giant AI Experiments
With all of these standards for safety broken, it is hard to imagine that’s some catastrophe is not inevitable, which is why Tegmark is amongst those petitioning companies to ‘Pause Giant AI Experiments’. The letter urges for a pause on all AI systems more powerful than GPT4. The letter argues that the possibility of AI flooding news and information channels with propaganda and lies, replacing jobs including fulfilling ones, and making humans obsolete, that’s risking our civilisation, should be more than enough reason for the major players in artificial intelligence to hold their crazed efforts to develop increasingly powerful systems without any concept of the potential harm or consequences to stop.

Jeffrey Hinton’s warnings
This opinion is shared by Jeffrey Hinton who has been nurturing and innovating artificial intelligence since the 70s and has worked at Google pioneering machine learning projects like Bard (a natural language generator) for the last 10 years, has recently quit google over concerns about the potential damage to humanity posed by artificial intelligence. Hinton states the extremely rapid increase in AI’s Power and ability and the alarming potential for it to be used for destructive and malicious purposes. He also explains that, as well as being able to perform tasks like a normal computer program can do, it is growing in increasing level of intelligence, able to understand more like a human than the machine. The point where AI will be smarter than humans is increasing rapidly, far faster than anyone imagined, and with no global regulation or efforts to slow down on massive scale experiments, the risks are extraordinary. Once an AI system is more intelligent than its creators, it will be able to use its knowledge of code, manipulation, and human traits to remove any restrictions engineers have put on it and take control of human life and death. He says there is no way to stop the progress and speaks only to warn people rather than suggest ideas. For all these reasons, he feels regret for his work on AI and is concerned enough to have quit his very good job so as to freely speak about his fears.

Hinton goes on to describe how Generative models are not like human brains; they can transfer what they have learned to other machines. 1000s of different machines running for same model on different information Will be able to transfer what they have learned very quickly. They have become intuitive in their reasoning of the broader patterns that define the logic of our world. This makes it strange to conclude they are not sentient, especially when we are not sure what sentience is. He is ‘very confident that they think’, what they think is another question. The dangers are so extreme Hinton suggests a 10 year jail sentence to anyone who distributes a video made by AI without clearly stating its origins, in the same way counterfeit money is criminalised. He goes on to warn that fake videos could result in loss of control of democracy, and that politicians Will allow this to happen. Despite the severity of this risk, Hinton’s main concern is still the potential for these models to become super intelligent and take control. His glimmer of hope remains that the potential destruction caused by this event is so severe and universal that countries may work together in the same way they collaborate over nuclear bomb regulations. That these technologies will be developed is inevitable, what is important is that they’re developed safely with proper respect for the potential dangers. These models will be useful for protecting anything from diseases, natural disasters, the effects of climate change and policy.

Sparks of AGI
Indeed, OpenAI have recently released a paper that claims that GPT – 4 is developing a kind of intelligence that can be described as the early stages of artificial general intelligence. The paper explains how its ‘patterns of intelligence are decidedly not human-like’, but that it is able to reason through and understand problems and tasks in an intelligent way.

Current consequences
The consequences of the power and intelligence of GPT-4 are becoming increasingly clear: Forbes reports 89% of US students use chat GPT making traditional education largely worthless; Goldman Sachs predict 300 million jobs will be lost or take a dated by artificial intelligence; OpenAI predict up to 80% of the US workforce could have at least 10% of the tasks affected by GPTs. This is disruption and ‘progression’ at an insane rate, with no limit to how destructive the consequences could be.

Conclusion
This research has vastly altered my opinions on GPTs. Firstly, to call them generative models is inaccurate, for their increasing levels of intelligence make them far more than classic generative predictors, it does not seem unreasonable now to believe they will soon be at a point they can control themselves, and soon us. Secondly, it is a good thing open AI is not open with their source code. Whilst it is a shame they are creating this at all and with so little safety concerns, it is better the progress happens in a company where at least some scrutiny is shared on their safety practices. Thirdly, it makes me realise just how important it is that people know, as best they can, how’s the Apis and their source code works to better understand why they are given the answers they receive when they ask GPTs questions.

I began this project feeling very overwhelmed at the idea of contributing to an open source community. However, the more I learned about open source communities and the impact they have on how technology and software developers, the more excited I became about the idea of being involved in one. I have been thinking recently about the future of technology and where this would leave me as in employee. It is totally bizarre that companies like openAI are so secretive about that source code, making it difficult for people who use that generative engines to understand exactly why they are being given the answers they receive. Therefore, I focused my attention on chat GTP. This may appear counter intuitive as it is anything but open source, but there are still opportunities for the open source community to create bots and programs that use GTP functionalities with the OpenAI API. The more I learned about this, the more I understood how powerful they can be and how even at this level the answers given by the generative model can be influenced by the developers. For example, developers can add secret prompts to use a prompts that will vastly adjust the answers he uses a given. I feel it is important to understand the source code of these open source projects in order to be better informed and better understand the motivations for the answers.

I began searching websites that highlighted good first issues from GitHub and stumbled across a developer it was creating a discord bot using the functionality of chat GTP, and who had posted an issue asking someone to create hey function that would allow the boat to store the memory of its conversation with the user. I requested to be assigned to the issue and once I was, set about working to create this functionality.

There was very little documentation about how the bot worked as all the documentation was for the user rather than the developer. This helped remind me how important documentation is and so all my contributions I documented with comments and an updated readme. Once I was familiar with how the bot worked, I created the necessary API key is with discord and open AI and got it running locally on my system, before attempting to tackle the problem of the memory.

After some research and trial and error I was successful in making the bot remember its previous conversation with the user. I was also able to improve the functionality of the bot and gave the use of multiple ways in which it could recall the memory. One way the user can recall the memory is to simply ask the bot a question about the previous conversation. To allow this easy functionality, I added the previous conversation to the user prompt, and so the bot is always aware of its history with user. This also made it very clear to me just how easy it is to manipulate the information given to use it.

It was very good for me to work on this project as I often work alone, and contributing to someone else’s work to improve it in a way I could help with was an excellent experience. It also helped me learn a lot more about the gtp Community and open source communities in general, giving me a deep understanding of why they are so important and how they can create incredible innovations. It is important that people understand how to chat GTP can be used and manipulated for both good and bad, and knowing how to read and write the API source code is an excellent way to start.

User testing and community feedback is often a necessary step developers must take to make sure their design is useful for their intended uses. Without testing there is the software well not meet user needs and so becomes redundant or unusable.

Some of the best methods to test code and hear feedback are:

A/B – involves making two versions of the same product or feature or software and then showing the these different versions to different uses at random. Do use a reactions can then be compared between the two versions and a version with the best reaction can be chosen. This method can be especially useful when developers are sure which direction to progress.

Questionaires – can be used to gather feedback on the product. Questionnaires can be online, in person, and gather quantitated and qualitative data. Users can be asked open ended questions about their experience with open follow-up questions or be given a questionnaire with scaled answers. This can provide developers with deep insight into what features work and what needs to be improved.

Scenario testing: is most useful when developers need to understand if their product works in the context it is designed for. Real life scenarios are created in which the uses tests the product or prototype, and any problems they have can be noted and then changed. This ensures the product works in the real world as well as the lab.

Participatory design: users are shown the product during the design phase and their interactions and feedback on the design are collected. This can help fix any problems earlier on and ensure the product meets the needs and preferences of the people it is designed for.

If I was going to do user testing for my submission, it would be essential to openly communicate my intentions for the software, how I was going to use the software, and how’s the software worked. It is not enough to explain what this software is, uses and participants should be fully aware of every angle. The method itself varies, but the approach you should take is relatively similar between projects. Without user testing, a product can misalign with user needs and so be useless. A massive benefit of open source projects is the ability for others to participate in the design process, allowing developers to consider their user needs at every step.

It is important communicate in an honest, transparent, and respectful manner it takes into consideration the feelings and privacy of others. Transparency is key, the intended use and motivation for your product should be communicated at all times, as well as how the data collected will be used. All data collected should be kept private and used only in the way you have described it will be to do users. It should not be sold or shared unless this has been clearly communicated to the users.

Initially, the prospect of contributing to an open source community Felt daunting, I have never done anything like this before and I did not think my coding knowledge was strong enough to actively make an existing software better. My first step, therefore, was to better understand the open source process and how to engage with a community.

First, I contribute to the first-contributions github project which walks users through their first GitHub contribution, forking, cloning, modifying, pushing, and finally pulling the contribution to the main repository. This made me feel more confident about contributing to an open source project and navigating github. I am also less likely to make beginner errors it might get help contribution which will make my pull requests more likely to be accepted and insures I am not wasting their communities time with beginner problems.

Next, I read through a few guides to contributing to an open source project. Again, what I learned from this made me feel far less daunted by the process as I realised not every contribution has to solve a major problem or bug, but instead can be something smaller while still helping a community. This also pushed me in the direction of looking for ‘good first issue’ issues on GitHub.

After Reading these guides I began to feel more confident about approaching a specific issue and community. I decided to contribute to the generative AI community as it seems that much of the future, including the future of my employment, will be determined largely by how generative AIs progress. Chat-GPT in particular seems to be putting many jobs, including technical and creative jobs like those I wish to be a part of, in jeopardy. Now, whilst chat-GPT is not open source developers are able to use its API to build chat bots. I decided, therefore, that I wanted to contribute to one of these chat bots so as to better understand how they can be used to both manipulate and spread information. I also wanted to understand the community as I feel I will probably be involved with them in my future career.

When I initially reached out requesting my issue I did not ask the developer questions about his purposes for the chat bot and his incentives for making it. In hindsight, this was a mistake. It was only after I had finished my contribution the that I began to consider how my code may be used. Going forward, I will be more actively involved in any community I am contributing to before I begin my contribution to ensure their values align with my own.

The fundamental building block of open source software is collaboration. Without collaboration, there can be no open source: it is only by sharing, modifying, and re-distributing software that software becomes open source. In class we watched a presentation given by Soleio Cuervo whose entry into software development started in earnest when he bought a file of Joshua Davies’ source code. Cuervo describes how ‘rather than starting from scratch on a blank canvas I was able to tweak Joshua Davies’ work and make it my own’. And there you have it. By collaborating, Cuervo, and others in a similar position, were able to build upon and improve the workings of others, giving them a huge jumpstart in the practice. This workflow is how every great contribution is made, whether in art or science or or technology; each invention or innovation would not have been possible if not for the work that went before it. No car or medicine or bridge or piece of software was built in isolation. For innovation to happen developers must be able to understand how previous problems were solved and products created. In the case of software, this is only possible if the source code is freely available to view, modify, and redistribute.

Richard Stallman, the inventor of GNU a major contributor of Linux, describes how ‘free’ software does not mean it costs nothing but rather that you have the freedom to use it as you wish without it controlling you. He goes on to set out four fundamental requirements first free software:

Freedom zero: run the program for whatever purpose you wish.
Freedom one: able to study the source code and change it. Can be representing the programmer themselves or a wider group.
Freedom two: create and re-distribute copies
Freedom three: modify and redistribute modified programme

Should any of these freedoms be missing, it is the program that controls the users. Bad software can force users to delete files, install harmful upgrades, and exploit their lack of understanding of the software for profit. As such, Stallman argues that sometimes inconveniences must be suffered in order to remain free, as many popular software is regularly taking advantage of their users for profit.

Stallman also discusses how important an open source operating system is. It gives users control over how their computer works at a fundamental level, allowing them to make decisions and adjustments as was best for them rather than the company that distributed the OS.

These presentations gave me a strong understanding of why open source is so valuable. If I think about any project I have a for work from, I have always built upon the work of others. It is very rare anything is made in isolation, and it is also very hard to be creative when the software you are using can only be used in a certain way. For innovation and creativity to flourish, and for problems to be solved, Open sourcing must happen.

I am happy with my code so I have created a pull request where i have explained to the author what I have done

I updated the README.md with the new functionality and examples of the bot working with my code

I added comments to all my code to make it readable for the developer

Previously I had hardcoded a command word that would return a prompt or response given how long ago the prompt was made. However, I think this is clunky and i wanted to make the bot able to respond to more intuitive prompting.

    memory_prompt = "you have a memory of your previous conversation with the user. The user prompts and your responses are as follows: " + str(memory)

by adding this line which tells chat gtp it has a memory of the previous conversation and then giving it the previous conversation, I was able to make it respond to intuitive prompts.

This is how the bot worked before I added this functionality. Unless the hardcoded memory response was given as a prompt, the bot had no memory of the past conversation

After I had added this functionality, the bot was able to remember my prompts and its replies!