I’m trying to stand up a Lemmy instance, and for some reason I’m just not getting it. I’ve got a fair bit of experience in Linux and Docker. NPM is new to me, but doesn’t seem difficult.

I’ve looked over several walkthroughs but it seems like they all don’t quite work right. Does someone have a clear step-by-step that works, or could take the time to remote in and help me get this up?

I’m running on VMWare ESXi, and I’ve tried both Debian and Ubuntu to get the server up. Closest I got, the Docker containers would start but seem to be throwing errors internally and don’t connect to one another.

    • TerryMathews@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 year ago

      I did. I could never get ansible to work when I was setting up the same machine. If you know how to set the inventory file up for that, I’m all ears.

        • ijustlookatpictures@feddit.uk
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          I’m currently hitting an issue of lets encrypt failing to authenticate using the .well-known. The domain in the hosts file is lemmy.domain.com though I have a feeling this may have to be the FQDN. the base domain is currently being used by matrix to serve antoher .well-known so it looks like I’ll have to add another page there somewhere.

      • fuser@quex.cc
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        assuming you’re setting up a dev instance from a linux PC, edit this line in hosts file:

        your_user@localhost domain=localhost letsencrypt_contact_email=you@yourdom.ain lemmy_base_dir=<path to web server root for lemmy>

        if you are running this at home and don’t have port 80 open you will also need to comment out the certbot/letsencrpyt lines from the lenny.yml file because certbot needs http to issue the cert. Create a self signed cert after the installation and edit the config file under sites_available to use the self signed cert instead of letsencypt.

        Lemmy works fine as a local dev instance with a self self-signed ssl cert and will subscribe and read from the fediverse without being able to post anything, of course because you don’t have a resolvable name, but it appears fully functional otherwise).

        chatGPT 3.5 is surprisingly familiar with lemmy’s configuration via ansible (thanks to all of reddit’s body of knowldege, presumably). So if you have a specific problem or immediate question, you can try that as well.

      • root@lemmy.belclayfer.net
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Ansible was a breeze once I got it going.

        I could never get ansible to work when I was setting up the same machine.

        Could you post your inventory file?

        • TerryMathews@lemmy.worldOP
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          So, I’m having what I assume is a very basic issue. Going through the ansible install steps, I’ve created an /etc/ansible/hosts with

          [servers] server1 ansible_host=localhost

          [all:vars] ansible_python_interpreter=/usr/bin/python3

          My understanding is this should work, but ansible all -m ping is failing.

        • TerryMathews@lemmy.worldOP
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Sorry that these are screenshots and not files, but I’m working through Chrome Remote Desktop so my options are a bit limited. This is what I get when I execute ‘ansible-playbook -i inventory/hosts lemmy.yml’ regardless of whether I have it set to terry@ or root@, and whether I use --become or not.

          • root@lemmy.belclayfer.net
            link
            fedilink
            English
            arrow-up
            1
            ·
            edit-2
            1 year ago

            Maybe you’ve already looked into this, but I checked the playbook and the error above is occuring when Ansible tries to run this command:

            test -e /usr/bin/python || (apt -y update && apt install -y python3-minimal python3-setuptools)

            If you manually run the part of the command in parentheses above you’ll probably get the Release file error also listed above, and from there you might be able to find out what’s up. I think that error usually has to do with your repository setup and/or the version of Ubuntu youre running, but Google can hopefully tell you for sure.

  • TerryMathews@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    So I wanted to make a top-level post: I’ve got a set of example files, and instructions, that will work 100% of the time on Debian. What do you guys think would be the best way to share them? A post here and the files shared on Google Drive? GitHub? Definitely open to suggestions, but I don’t want anyone to struggle with it as hard as I have.

    • ThorrJo@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      oh man. super valuable. I would love to have that, as setting up Lemmy on Debian 12 is in my near future.

      • TerryMathews@lemmy.worldOP
        link
        fedilink
        English
        arrow-up
        5
        ·
        edit-2
        1 year ago

        Here we go:

        Example files

        Any place you see <something>, you need to change it to fit and omit the <>. If something <matches> in two differet places <matches> like this, make sure they match when you’re done as well. Specifically, the postgres user and password in the lemmy docker file and the lemmy.hjson.

        Finally, in Google drive the files end in .txt so you can view them. You’ll need to correct the file names when you download them if you intend to use them. You should have two docker-compose.yml, one in each of the two directories you create, and one lemmy.hjson.

        From a fresh CLI Debian 11 install:

        su
        /sbin/usermod -aG sudo <user>
        groups <user>
        apt-get install sudo
        cd /opt
        mkdir npm
        cd npm
        (copy or create docker-compose.yml)
        apt-get install docker-compose
        docker-compose up -d
        cd /opt
        mkdir lemmy
        cd lemmy
        (copy or create docker-compose.yml and lemmy.hjson)
        mkdir -p volumes/pictrs
        chown -R 991:991 volumes/pictrs
        docker-compose up -d
        docker ps (verify containers are all running, grab ip address for lemmy container)
        Configure port forwarding in npm for your lemmy container (npm should be accessible at debian_ip_address:81)
        Remember to do the custom paths from the various guides. The lemmy port in this guide is 1234.
        

        Please note I am not addressing federation or SSL or true hosting yet. I haven’t got that far yet. But if you can get the damn thing running, the last mile shouldn’t be too bad.

        • ThorrJo@lemmy.sdf.org
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          Thank you! This is helpful.

          I will probably be installing it bare metal atop Debian (no Docker), but this is still quite useful.

          BTW I think if you indent your list of commands by 4 spaces, it will render better:

          su
          /sbin/usermod -aG sudo <user>
          groups <user>
          apt-get install sudo
          cd /opt
          mkdir npm
          cd npm
          (copy or create docker-compose.yml)
          apt-get install docker-compose
          docker-compose up -d
          cd /opt
          mkdir lemmy
          cd lemmy
          (copy or create docker-compose.yml and lemmy.hjson)
          mkdir -p volumes/pictrs
          chown -R 991:991 volumes/pictrs
          docker-compose up -d
          docker ps (verify containers are all running, grab ip address for lemmy container)
          
  • sven@l.mchome.net
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    Hopefully this will help someone. This seems to work for me. Subscribed communities update, I am able to post. I’m the only user right now on my server. NPM took me a bit of messing around with the config but I think I have everything working, some of this may be redundant / non functional but I don’t have the will to go line by line to see what more I can take out. Here is how I have it configured. Note that some things go to the Lemmy UI port and some to the Lemmy port. These should be defined in your docker-compose if you’re using that. (Mine is below)

    On the first tab in NPM, “Details” I have the following:

    Scheme: http
    Hostname: <docker ip>
    Port: <lemmy-ui port>
    Block Common Exploits and Websockets Support are enabled.
    

    On the Custom Locations page, I added 4 locations, you have to do one for each directory even though the ip/ports are the same.

    Location: /api
    Scheme: http
    Hostname: <docker ip>
    Port: <lemmy port>
    

    Repeat the above for “/feeds”, “/pictrs”, and “/nodeinfo”. The example file they give also says to have “.well_known” in there but as far as I know that’s just for Let’s Encrypt which NPM should be handling for us.

    On the SSL tab, I have a Let’s Encrypt certificate set up. Force SSL, HTTP/2 Support, and HSTS Enabled.

    On the Advanced tab, I have the following:

     location / {
    
       set $proxpass "http://<docker ip>:<lemmy-ui port>";
       if ($http_accept = "application/activity+json") {
    
         set $proxpass "http://<docker ip>:<lemmy-ui port>";`
       }
       if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
         set $proxpass "http://<docker ip>:<lemmy-ui port>";
       }
       if ($request_method = POST) {
         set $proxpass "http://<docker ip>:<lemmy-ui port>";
       }
       proxy_pass $proxpass;
       
       rewrite ^(.+)/+$ $1 permanent;
        # Send actual client IP upstream
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     }
    

    I probably should add in my docker compose file as well… I’m far from a docker expert. This is reasonably close to their examples and others I found. I removed nginx from in here since we already have a proxy. I disabled all the debug logging because it was using disk space. I also removed all the networking lines because I’m not smart enough to figure it all out right now. If you use this, look out for the < > sections, you need to set your own domain/hostname, and postgres user/password.

    version: "3.3"
    
    services:
      lemmy:
        image: dessalines/lemmy:0.17.3
        hostname: lemmy
        restart: always
        ports:
          - 8536:8536
        environment:
          - RUST_LOG="warn"
          - RUST_BACKTRACE=full
        volumes:
          - ./lemmy.hjson:/config/config.hjson:Z
        depends_on:
          - postgres
          - pictrs
    
      lemmy-ui:
        image: dessalines/lemmy-ui:0.17.4
        # use this to build your local lemmy ui image for development
        # run docker compose up --build
        # assuming lemmy-ui is cloned besides lemmy directory
        # build:
        #   context: ../../lemmy-ui
        #   dockerfile: dev.dockerfile
        ports:
          - 1234:1234
        environment:
          # this needs to match the hostname defined in the lemmy service
          - LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy:8536
          # set the outside hostname here
          - LEMMY_UI_LEMMY_EXTERNAL_HOST=< domain name>
          - LEMMY_HTTPS=false
          - LEMMY_UI_DEBUG=true
        depends_on:
          - lemmy
        restart: always
    
      pictrs:
        image: asonix/pictrs:0.4.0-beta.19
        # this needs to match the pictrs url in lemmy.hjson
        hostname: pictrs
        # we can set options to pictrs like this, here we set max. image size and forced format for conversion
        # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
        environment:
          - PICTRS_OPENTELEMETRY_URL=http://otel:4137
          - PICTRS__API_KEY=API_KEY
          - RUST_LOG=debug
          - RUST_BACKTRACE=full
          - PICTRS__MEDIA__VIDEO_CODEC=vp9
          - PICTRS__MEDIA__GIF__MAX_WIDTH=256
          - PICTRS__MEDIA__GIF__MAX_HEIGHT=256
          - PICTRS__MEDIA__GIF__MAX_AREA=65536
          - PICTRS__MEDIA__GIF__MAX_FRAME_COUNT=400
        user: 991:991
        volumes:
          - ./volumes/pictrs:/mnt:Z
        restart: always
    
      postgres:
        image: postgres:15-alpine
        # this needs to match the database host in lemmy.hson
        # Tune your settings via
        # https://pgtune.leopard.in.ua/#/
        # You can use this technique to add them here
        # https://stackoverflow.com/a/30850095/1655478
        hostname: postgres
        command:
          [
            "postgres",
            "-c",
            "session_preload_libraries=auto_explain",
            "-c",
            "auto_explain.log_min_duration=5ms",
            "-c",
            "auto_explain.log_analyze=true",
            "-c",
            "track_activity_query_size=1048576",
          ]
        ports:
          # use a different port so it doesnt conflict with potential postgres db running on the host
          - "5433:5432"
        environment:
          - POSTGRES_USER=< dbuser >
          - POSTGRES_PASSWORD=< dbpassword>
          - POSTGRES_DB=lemmy
        volumes:
          - ./volumes/postgres:/var/lib/postgresql/data:Z
        restart: always
    

    There’s another post over here https://fernchat.esotericmonkey.com/post/277 with a similar setup, less stuff in the Advanced tab. If I’m bad at explaining maybe another guide will help.

    • TerryMathews@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      This worked for me, with one note:

      <dbuser> and <dbpassword> need to be lemmy and password if you’re using the stock lemmy.hjson file, or the lemmy_lemmy_1 container will get stuck in a reboot loop. There’s no define in the stock file in githubusercontent currently for those, so you have to add them by hand.

      For security, you have to change these.

  • Slashzero@hakbox.social
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    Hi there! What part of the docker setup are you getting hung up on? Can you post the log errors?

    sudo docker logs lemmy_lemmy_1

    And you can replace lemmy_lemmy_1 with the other service names for their logs.

    To list the running docker containers (services):

    sudo docker container ls or sudo docker ps

  • mo_ztt ✅@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    I just set it up from a git checkout, and it was pretty involved – the build instructions are actually quite a bit better than usual for development software at this stage, but it’s still a complicated process with some changes vs. what’s in the instructions. If you’re open to that route, I can try to document what I did in more detail + send it along and give some help if you get stuck.

    • giddy@lemmy.perthchat.org
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      1 year ago

      Not the OP but I would appreciate it. I got stuck with building the lemmy image. Not sure why they don’t just use a published image

      EDIT - found the problem - the docker compose file has the build options enabled for the lemmy image and the published image commented out

      EDIT 2 - latest issue - 404: FetchError: request to http://lemmy:8536/api/v3/site? failed, reason: getaddrinfo EAI_AGAIN lemmy

      • seang96@exploding-heads.com
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        I had an AI write 95% of my kurbenetes config. I had this at one point, it is likely you are not proxying (usually nginx) the Lemmy backend to /api and there are are other endpoints that need mapped to it too.

    • TerryMathews@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      That would be great. I’m not opposed to building, I was actually just sitting here contemplating deep-sixing the whole docker arrangement and just building the services up. That’s what I learned back in the day, and to me it feels like Docker and having everything as separate images is just giving me an additional point of failure.

      • mo_ztt ✅@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Sure thing, I’ll put together some notes (or more likely some proposed revisions to the “how to build from source” that’s on join-lemmy.org) in the next few days I think

  • jamesravey@lemmy.nopro.be
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    1 year ago

    Hey - I found the same thing WRT the docker files - the compose files from the official project are ever-so-subtly wrong.

    Tagging a docker network as internal blocks outside network comms afaik so the default compose file essentially puts the lemmy server inside its own little sandbox and prevents it from communciating with other servers.

    The solution I found was to add lemmy to both the internal network and the external proxy network:

    
    ## this is what the networks part looks like by default
    networks:                                                                                                                                                   
      # communication to web and clients                                                                                                                        
      lemmyexternalproxy:                                                                                                                                       
      # communication between lemmy services                                                                                                                    
      lemmyinternal:                                                                                                                                            
        driver: bridge                                                                                                                                          
        internal: true            
    
    #... other stuff here
    #lemmy service inside your services: section
      lemmy:
        image: dessalines/lemmy:0.17.3
        hostname: lemmy
        networks:
          - lemmyinternal
          - lemmyexternalproxy # this is the important addition
        restart: always
        environment:
          - RUST_LOG="warn,lemmy_server=info,lemmy_api=info,lemmy_api_common=info,lemmy_api_crud=info,lemmy_apub=info,lemmy_db_schema=info,lemmy_db_views=info,l
    emmy_db_views_actor=info,lemmy_db_views_moderator=info,lemmy_routes=info,lemmy_utils=info,lemmy_websocket=info"
        volumes:
          - ./lemmy.hjson:/config/config.hjson
        depends_on:
          - postgres
          - pictrs
    
    

    Another thing I noticed was that in the documentation they bind nginx on port 80 but the docker-compose provided binds to port 8536 which is the default port that lemmy seems to listen on. I bound 8536 to my host machine and use caddy as a reverse proxy (because it does letsencrypt for you which is nice).

    (Writing to you now from my self-hosted instance which I set up with the above notes)

  • nii236@lemmy.jtmn.dev
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    I tried many methods but ended up having to use Ansible. That worked without a hitch.

    Sucks I had to spin up a new server for it though but that’s the cost of decentralisation I guess