In visual art these days, creativity has no limits.

The latest technology really does let the imagination run wild. Whatever you can think of, AI will help you translate it into a visually stunning reality. 

‘SnoopStein’ dabbles with DALL-E

If you’re familiar with Midjourney—the AI service that lets you create images with simple prompts—think of the project below as a way to build your very own Midjourney. Except this version makes use of our celebrity Twitter bot SnoopStein, a mashup of rapper Snoop Dogg and Albert Einstein. 

Our version also makes use of DALL·E 3, from OpenAI, along with Twitter, and you’ll pull everything together in MindsDB.

What does MindsDB do?

The MindsDB platform connects real-time data to AI, and lets companies prototype and deploy AI-powered enterprise solutions quickly and at scale. In this case, MindsDB will act as an orchestration layer between Twitter and DALL·E 3. 

MindsDB makes building AI into applications easy for developers at all skill levels. Because MindsDB brings AI to the data layer, you can manage everything via SQL. SQL is the most common language developers use to manage data, but MindsDB modified its syntax to also build AI workflows. 

No SQL? Never fear

Knowing a few SQL commands will come in handy, but no worries if not. You can also build the same using MindsDB via REST API endpoints, Mongo-QL, or directly from your Python or JavaScript code. 

If you get stumped, ask us a question through Community Slack. 

Study up on virtual tables

Before we begin the tutorial, it’ll help to familiarize yourself with the concept of AI Tables. This is the way MindsDB abstracts models as virtual tables in the database. In the following tutorial you’ll see a lot of Create Model statements, which does exactly this—represent AI models as virtual tables inside databases. So that you can query, join and manipulate AI data exactly as if you deal with regular data.

MindsDB has hundreds of integrations, and here we’ll make use of one between MindsDB and Twitter. 

Make your own Midjourney in 3 steps

So let’s begin. Here’s the step-by-step guide to building your custom Midjourney-like bot on Twitter.

‍

1. Create OpenAI models with a bit of personality. 

First let’s create models in MindsDB for generating text and images. These models act as AI tables and can be queried with SQL.

In order to create a model, you’ll need an OpenAI account and an API key. You’ll also need a MindsDB installation—you can find an open-source version here.

Then go to your MindsDB SQL Editor and enter the following commands: 

‍

I. Model to generate a text response:

Before creating a model, please first create an engine (in our case based on OpenAI), providing your OpenAI API key:

CREATE ML_ENGINE openai_engine
FROM openai
USING
   api_key = 'sk-xxx';

Now you can create a model:

CREATE MODEL mindsdb.snoopstein_describe_image_model
PREDICT response
USING
engine = 'openai_engine',
max_tokens = 300,
model_name = 'gpt-4', -- you can also use 'text-davinci-003' or 'gpt-3.5-turbo'
prompt_template = 'From input message: {{text}}\
by from_user: {{author_username}}\
In less than 250 characters, write a Twitter response to {{author_username}} describing the image
 they want to see; respond a rhyme as if you were Snoop Dogg but you also were as smart as Albert
 Einstein, still explain things like Snoop Dogg would, do not mention that you are part Einstein.
 If the question makes no sense, explain that you are a bit lost, and make something up that is
 both hilarious and relevant.';
 

The CREATE MODEL command creates and deploys the model within MindsDB. Here we use the OpenAI GPT-4 model to generate text responses to users’ questions. The prompt_template message sets the personality of the bot—here, SnoopStein.

Please note that the prompt_template message contains two variables: {{text}} and {{author_username}}. These variables are replaced with the text and author_username column values from the table that stores tweets. The replacement happens upon joining the model table with the tweets table.

Let’s test it:

SELECT response
FROM mindsdb.snoopstein_describe_image_model
WHERE author_username = "someuser"
AND text="@snoop_stein, why is gravity so different on the sun?";
 
 

Here is a sample reply from the model:

II. Model to generate an image response:

We’ll use the latest OpenAI DALL·E 3 AI model to generate images as part of the responses.

CREATE MODEL mindsdb.snoopstein_image_model
PREDICT img_url
USING
engine = 'openai_engine',
mode = 'image',
model_name = 'dall-e-3',
prompt_template = 'Make a photorealistic image. Here is the description: {{response}}, 4k, 
digital painting';
 

The CREATE MODEL command creates and deploys the DALL-E 3 model within MindsDB. This model will generate images based on SnoopStein’s text response (from the first model). 

The prompt_template message contains the {{response}} variable. This variable is replaced by the prediction of the previous model upon chaining the two models.

Let’s test it:

SELECT textresponse.response, image_response.img_url
FROM (
    SELECT response
    FROM mindsdb.snoopstein_describe_image_model
    WHERE author_username = "someuser"
    AND text="@snoop_stein, why is gravity so different on the sun?") AS textresponse
JOIN mindsdb.snoopstein_image_model AS image_response;
 
 

Here is a sample reply:

The DALL·E 3 model provides a link to the generated image.

2. Set up your Twitter dev account and connect it to MinsdDB.

Now we can proceed with building a process of tracking new tweets that mention @Snoop_stein (or your own Twitter account) and posting responses on its behalf using model-generated content.

You will need to sign up for a Twitter dev account in order to actually be able to read and write tweets into Twitter, if you don’t have one already.

Twitter may take a day or so to approve your new dev account.  Once you are approved, here are the steps to link your Twitter account to MindsDB: https://docs.mindsdb.com/integrations/app-integrations/twitter

Here is the command that connects your Twitter account to MindsDB:

CREATE DATABASE my_twitter
WITH
  ENGINE = 'twitter',
  PARAMETERS = {
     "consumer_key": "todo",
     "consumer_secret": "todo",
     "bearer_token": "todo",
     "access_token": "todo",
     "access_token_secret": "todo"
  };
 

Check out this usage guide to learn how to query and insert tweets from MindsDB.

3. Automate the Twitter bot with MindsDB.

The following process runs on schedule. First MindsDB fetches all new tweets with a mention of @snoop_stein, then it retrieves both text and image from OpenAI. Then it posts a response tweet tagging the user who asked the question.

In MindsDB you can schedule query execution using JOBS.

CREATE JOB twitter_bot_snoopstein_updated (
 
   INSERT INTO my_twitter.tweets (in_reply_to_tweet_id, text, media_url)
      SELECT textresponse.id AS in_reply_to_tweet_id,
             concat(textresponse.response, chr(10), chr(10), 'VOILA G!!') AS text,
             image_response.img_url AS media_url
      FROM (
              SELECT t.id AS id, t.text AS text, m.response AS response
              FROM my_twitter.tweets AS t
              JOIN snoopstein_describe_image_model AS m
              WHERE 
                 t.query = “(@snoopstein OR @snoop_stein 
                            OR #snoopstein OR #snoop_stein) -is:retweet -from:snoop_stein” 
                 AND t.id > LAST
           ) AS textresponse
      JOIN snoopstein_image_model AS image_response
 
) EVERY 2 minutes;

Within parentheses, provide all statements to be executed by the job. In this tutorial, we’ve set it to run once every two minutes, but you can define your own interval.

This job inserts replies to tweets into the internal MindsDB tweets table (MindsDB’s Twitter integration will then post these inserts as normal tweets automatically). We provide the SELECT statement as an argument to the INSERT statement. Note that the inner SELECT statement uses one model to generate a text response (aliased as textresponse). 

Then, the output is joined with another model that generates an image (aliased as image_response) based on the text response generated by the first model.

You can monitor this job with the following commands:

SELECT * FROM jobs
WHERE name='twitter_bot_snoopstein_updated';
 
SELECT * FROM jobs_history
WHERE name='twitter_bot_snoopstein_updated';

Here is a sample reply:

Once you’ve automated the job, the only limit is your imagination. Generate any image you like using MindsDB, DALL·E 3, and a few simple SQL commands.

Extra credit and next steps

Want to build a similar bot but interact with it via text message? Check out our tutorial on how to build an SMS AI agent using a MindsDB-Twilio integration. In fact, MindsDB can help you build a similar bot or agent using Slack, Rocket Chat, email or any other messaging platform. 

The workflow is basically the same. The only thing you change is the integration to read and insert user messages.

You can also change the bot’s personality to suit your setting. Need images for a homemade hip-hop video? Or your children’s book could use some illustrations? Switch the prompt_template message in the tutorial above to channel Dr. Dre or Dr. Seuss, accordingly. 

Let us know what you come up with, and if you need any help. MindsDB is open-source and it has a large, responsive Slack community where builders exchange ideas. We’d also love to hear what you’ve built. 

Good luck with your AI projects.