• sunzu@kbin.run
    link
    fedilink
    arrow-up
    14
    ·
    3 months ago

    Do we know if local models are any safer or is that a trust me bro?

    • David Gerard@awful.systemsOPM
      link
      fedilink
      English
      arrow-up
      27
      ·
      3 months ago

      well we’re talking about data across a company. Tho apparently it does send stuff back to MS as well, because of course it does.

      • SurpriZe@lemm.ee
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        1
        ·
        3 months ago

        Best way to deal with it? What’s the modern solution here

        • self@awful.systems
          link
          fedilink
          English
          arrow-up
          23
          ·
          3 months ago
          • don’t use any of this stupid garbage
          • if you’re forced to deploy this stupid garbage, treat RAG like a poorly-secured search engine index (which it pretty much is) or privacy-hostile API and don’t feed anything sensitive or valuable into it
          • document the fuck out of your objections because this stupid garbage is easy to get wrong and might fabricate liability-inducing answers in spite of your best efforts
          • push back hard on making any of this stupid garbage public-facing, but remember that your VPN really shouldn’t be the only thing saving you from a data breach
          • SurpriZe@lemm.ee
            link
            fedilink
            English
            arrow-up
            6
            arrow-down
            1
            ·
            3 months ago

            Thanks but it’s too late. Here it’s all over unfortunately. I’m just doing my best to mitigate the risks. Anything more substantial?

            • froztbyte@awful.systems
              link
              fedilink
              English
              arrow-up
              8
              ·
              3 months ago

              “better late than never”

              if it already got force-deployed, start noting risks and finding the problem areas you can identify post-hoc, and speaking with people to raise alert level about it

              probably a lot of people are going to be in the same position as you, and writing about the process you go through and whatever you find may end up useful to others

              on a practical note (if you don’t know how to do this type of assessment) a couple of sittings with debug logging enabled on the various api implementations, using data access monitors (whether file or database), inspecting actual api calls made (possibly by making things go through logging proxies as needed), etc will all likely provide a lot of useful info, but it’ll depend on whether you can access those things in the first place

              if you can’t do those, closely track publications of issues for all the platforms your employer may have used/rolled out, and act rapidly when shit inevitably happens - same as security response

              • SurpriZe@lemm.ee
                link
                fedilink
                English
                arrow-up
                2
                ·
                3 months ago

                How’s it at your place? What’s your experience been with this whole thing

                • froztbyte@awful.systems
                  link
                  fedilink
                  English
                  arrow-up
                  8
                  ·
                  3 months ago

                  whenever any of this dogshit comes up, I have immediately put my foot down and said no. occasionally I have also provided reasoning, where it may have been necessary/useful

                  (it’s easy to do this because making these calls is within my role, and I track the dodgy parts of shit more than anyone else in the company)

                  • SurpriZe@lemm.ee
                    link
                    fedilink
                    English
                    arrow-up
                    2
                    ·
                    3 months ago

                    Hm, that’s good to have such authority on the matter. What’s your position?

                    I’m struggling with people who don’t fully understand what this is all about the most.

            • MonkderVierte@lemmy.ml
              link
              fedilink
              English
              arrow-up
              4
              ·
              3 months ago

              Limit access on both sides (user and cloud) as far as you can, train your users if possible. Prepare for the fire, limit liability.

    • BlueMonday1984@awful.systems
      link
      fedilink
      English
      arrow-up
      11
      ·
      3 months ago

      Local models are theoretically safer, by virtue of not being connected to the company which tried to make Recall a thing, but they’re still LLMs at the end of the day - they’re still loaded with vulnerabilities, and will remain a data breach waiting to happen unless you make sure its rendered basically useless.

    • sturlabragason@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      7
      ·
      edit-2
      3 months ago

      You can download multiple LLM models yourself and run them locally. It’s relatively straightforward;

      https://ollama.com/

      Then you can switch off your network after download, wireshark the shit out of it, run it behind a proxy, etc.