Get started with App.net.

  Sign up now  
keita
    Ugh, I want elasticsearch to output its logs in JSON, but I’m having no luck at all with it.
    keita
      @matigo actually, SQL Server (and Oracle for that matter)’s RDS cost doesn’t add much of an overhead, all things considered
      keita
        @matigo or, just use a better database
        keita
          @matigo ugh, yes, I know about that trick. unfortunately, can’t do that on RDS
          keita
            The way how MySQL puts almost everything in to disk-based temporary tables is really infuriating
            keita
              I’m getting good at this fluentd stuff
              keita
                The biggest benefit for going wireless for headphones for me is that I can connect them to the TV now.
                keita
                  @cgiffard that means you need infinity storage :P
                  keita
                    @cgiffard yikes, petabyte scale.
                    keita
                      @cgiffard How big is your cluster? (just wondering what kind of scale you’re seeing these problems at)
                      keita
                        @cgiffard Fortunately we haven’t had to do any massive tuning yet, but we’ll see
                        keita
                          @cgiffard (there were a few cases of putting more than 32kb in a not_analyzed string field)
                          keita
                            @cgiffard Yeah — also, thanks to Athena we can query against data stored in S3 as well :)
                            keita
                              @cgiffard (we’ve had problems with logs not going in to ES before because the format conflicted with the index and was just dropped — something that monitoring ES’s logs would have caught)
                              keita
                                @cgiffard everything that is a log, goes to ES. Even ES’s logs. (they go to S3 as well as a failsafe if ES is down, in addition to fluentd’s buffers)
                                keita
                                  @cgiffard FEK (s/logstash/fluentd/g), haha. Was considering logstash, but we already have a considerable amount of fluentd log shipping stuff
                                  keita
                                    All of your logs are belong to us!