I know memory is fairly cheap but e.g. there are millions of new videos on youtube everyday, each probably few hundred MBs to few GBs. It all has to take enormous amount of space. Not to mention backups.

  • assembly@lemmy.world
    link
    fedilink
    arrow-up
    26
    arrow-down
    4
    ·
    1 year ago

    It’s the same story with AWS as well. They use vast amounts of storage and leverage different tiers of storage to get the service they want. It’s funny but they have insane amounts of SD cards ( cheapest storage available at the size) and use that for some storage and just replicate things everywhere for durability. Imagine how small 256GB SD cards are and they you have hardware to plug-in 200 of them practically stacked on top of each other. The storage doesn’t need to be the best, it just needs to be managed appropriately and actively to ensure that data is always replicated as devices fail. That’s just the cooler tier stuff. It gets complex as the data warms.

      • asteroidrainfall@kbin.social
        link
        fedilink
        arrow-up
        7
        ·
        1 year ago

        Yeah this seems false. SD cards are unreliable, hard to keep track of, and don’t actually store that much data for the price. I do think they use tapes though to store long term, low traffic data.

        • assembly@lemmy.world
          link
          fedilink
          arrow-up
          2
          ·
          1 year ago

          Used to work there but it’s been a few years so maybe things changed but that was how we originally got super cheap and durable s3.

        • zalack@kbin.social
          link
          fedilink
          arrow-up
          1
          ·
          edit-2
          1 year ago

          We use LTO tapes in Hollywood to back up raw footage; it wouldn’t surprise me if AWS uses tapes for glacier.

          I got a tour of Iron Mountain once (where we sent tapes for long term archival). They had a giant room with racks and racks of LTOs, and a robot on rails that would make copies of each tape at regular intervals to keep the data from corrupting. It looked kinda like the archive room in Rogue One. Wouldn’t surprise me if Iron Mountain was an inspiration for the design. Super interesting.

          • LostXOR@kbin.social
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            That sounds really cool! I wonder what sort of redundancy they have in case a tape gets damaged or corrupted.

    • vyvanse@kbin.social
      link
      fedilink
      arrow-up
      5
      ·
      1 year ago

      Ha, I had no idea data centers use SD cards! It makes sense in hindsight, but it’s still funny to think about