melannen | (no subject)

I am catching up on the last two weeks of DW using the reading-list-by-date option, and it is doing weird things. The sort of weird things that make me doubt my reality. I have noticed this before when doing long catch-ups, but have never been awake enough at the time to be quite certain it's real.

Basically, I am paranoid that I am missing posts from people.

Like

denise's post on pseudonymity? I saw that because someone else on my list linked to it, I don't remember seeing it come up on the list. Which could be just the fact that I do, inevitably, skim a little bit, even when I'm not meaning to, and I could be skim-missing posts that I really, really would rather not.

But.

Then there is this: I go to a day, and then I skip to the beginning of that day's personal-journal posts until I am no longer given an option to go to previous 20 posts; this usually lands me at skip=40 or skip=60. Usually, this oldest page has substantially fewer than 20 posts in it - presumably because the number of posts in a day is not evenly divisible by 20, fine. But then, when I get to the last page in the day, which should show the most recent 20 posts, there is a certain amount of overlap with the previous page, the skip=20 one.

This confuses me. Okay, there is overlap because the posts aren't evenly divisible by 20! But... we took care of that with the first page being less than 20. I am confused. But I can deal; there's some weird stuff going on behind the scenes, fine.

But then! The problem is, the overlap isn't complete. For example, I just finished the page http://melannen.dreamwidth.org/read?show=p&skip=20&date=2011-08-07 , and am moving on to the page http://melannen.dreamwidth.org/read?show=p&date=2011-08-07 . Currently, there is a post by copperbadge 8 from the top of the skip=20 page. That same post by copperbadge is the bottom post on the skip=0 page. However, on skip=20, the next 8 posts are (encoded for privacy): n1, j1, j2, l1, d1, d2, s1, l2. On skip=0, the posts following the copperbadge post are l1 and s1, and n1, j1, j2, d1, d2, and l2 do not appear at all on that page.

This confuses me lots. And it's not just those pages: it's happening on most of the days. I would ask you guys to go and see if the same thing happens to you on those pages, but I don't think they would look the same to anyone not logged in as me. However, I can ask: has anyone noticed this same issue and can back me up? Is it a known bug?

There is no obvious pattern to which posts get skipped in the overlap (locked/unlocked, cut/not cut, certain users always, etc.) but I haven't looked too deeply. I'm mainly paranoid that there are other random posts getting skipped, and I'm only noticing the overlapped ones.

(I will clean this up and post to Support about it, probably, once I've come to the end of my backlog, I just wanted to document it now in case I forget. And hopefully get somebody confirming that it's not just me imagining things, or just my deeply broken style that I still haven't fixed.)

Flat | Top-Level Comments Only

I don't catch up by date so I haven't seen that specifically, but I've seen something similar. When I skip back through my network pages, some things that I saw at the bottom of skip=20 may be at the top of skip=40. Fine, more people on my network have posted, right? Except that where the sequence on skip=20 was C, D, E, F, when I skip back another page it may be D, R, E, F at the top -- ie, posts are adding into the middle, what looks like out of order. Now I'm wondering if I'm missing posts because I skip back until I recognise a post -- but what if there are things I haven't seen after that point?

I went back to network skip=300 last night, looking for something I'd seen, and I spotted at least four or five posts I had not seen before. I didn't turn up the post I was looking for, either -- I'd assumed I lost patience before I went back far enough, but now I wonder. (It was a rec post; I wouldn't have thought there was a reason to lock it.)

That sounds like exactly the same issue I'm having, the two of us are just skipping in the opposite order. Good! Well, not good, but confirmation that it's not just me.

Also, yes, I've also done the thing where I swear I saw a post, but when I skip back I can't find it: I've just been assuming it was retro-actively locked, but...

Skipping/adding in would be the same thing from oppostie directions, wouldn't it? ::facepalm::

I have terrible time-sense for "when did I see that," so I thought it was me, but the rec post that vanished I am nearly *sure* was just over the weekend...

I have this with my network page, too.

Okay, definitely a more widespread issue then...

I was thinking it was like a post being started with an earlier timestamp and the person not updating the timestamp posting it, which gets it sorted into the rest kind of unofficially backdated. Which I can't really prove, though, because I haven't noticed it, but the new posts appearing before posts I have read happens to me too.

That wouldn't really apply with what I've noticed, because weeks-old reading pages are going to be pretty static (Even if people backdate that far, it doesn't turn up on the rlist; plus I have reloaded the pages multiple times and the skip is still appearing.)

It's possible that does explain some of the network stuff though. Hmm.

I haven't delved into network past a couple of days, so I don't know, hmm. I am starting to chase down what may be an issue with timestamp-updating, too.

Yeah, this happens to me on my reading list, too.

I used to notice it on LJ as well.

The network page is a little weird like that -- it will repeat posts fairly regularly (it has to do with a combination of handwave handwave and the way a reading page is generated and the algorithms involved in producing the set of "people who are read by the people I am reading" and the caching being used so that it doesn't overload the databases). It shouldn't lose posts, but it will frequently duplicate them. (Essentially, the problem happens when someone is read by multiple people on your reading list.)

The ?date=foo reading page may or may not have similar issues. I'm not sure; we haven't done a lot of looking into that.

We had a problem in early beta where we did some optimization work and posts started disappearing on the reading list -- we backed out the change and it seemed to have fixed itself, but if it's manifesting again, it might be related to something else. We've been pulling webservers in and out of production and messing around with the network, though, and that might cause problems with the handwave handwave and the memcache. If it continues past the next week or two, then it might be time to look into it; if it's only been happening recently, though, it's probably due to the way we've been popping webservers in and out of the pool based on time and load.

I can now confirm that the pseudonymity post definitely did *not* appear on my by-date reading lists, and with majoline, that's two, there.

I honestly don't know when it started for me; I know I've been wondering about weirdnesses like that since at least the last time I did a long catch-up (possibly Christmas? definitely before the current troubles) and there've been occasional posts, for awhile, that I should have seen on my reading list but didn't but assumed it was my fault, but I don't have any actual data from before now, so I cannot be sure.

Also officially missing:

hradzka's post on the Breivik manifesto, from 8/10.

Also, when I got to the last page of by date (8/17), it was about four hours behind where my reading list is (possibly this is a known behaviour? in which case, hmmm.)

8/16 is another day with many skips between pages.

Huh. I've never had a post show up twice. What I am seeing is very specifically posts turning up in the middle of things where they were not before, past the point where I'd think "Oh, they just took a few minutes to hit post." (So presumably those posts got skipped before?)

It's been going on longer than the webservers issue -- I can't pin it down to a date or anything, but I'd taken it as a normal thing while back-reading, as new posts were added to the most recent page. It definitely didn't suddenly start recently.