New API questions

I'm currently accessing Danbooru completely through its APIs (to extend the functionality of the subset of features I use, and to standardize UI for several boorus).
However I have been unable to find much information about the new API. I have found that the /X/index.xml are now /Xs.xml, however I still don't know what parameters to pass them nor how /X/Y.xml are accessed. (I don't really use those anyway though.)

I have been unable to figure out how to look up a tag. /tags.xml?name=X doesn't work and /tags.xml?search=X errors (HTTP error 500).
I'm also a bit confused with the data. One tag has a name of "parent:712273", a post-count of 0 yet still have related tags. Another tag has a name of "tag:gypsy". (I could not find any such tags in my old copy of Danboorus tags either.)
/related_tag.json also gives me a error 500.

What became of the post count in post(s).xml? Not being able to know the about of posts is quite inconvenient for me. First of all I wouldn't be able to show a correct amount of pages, but more importantly it is one of the few parameters I can use to control my partial cache.
(The post count is the main reason I have been using the XML versions.)

On a positive note, the fact that the new API appears to have a way of showing which pools a post is in makes me quite excited, as it was the only thing I missed. Why is they prefixed with "pool:" (and "fav:" in fav-string) though? Can they have different prefixes in certain situations?
I don't quite understand why there are tag-counts for the various tag types. If the tag-string was ordered by tag type instead of just alphabetically you could use it to figure out which tags where which, but right now I don't see the point. (Also fav-count seems always to be 0, no matter how long fav-string is?)

EDIT: Also how are the image links? I imagine it is something like this:
thumb: /ssd/data/preview/[md5].jpg
sample: /data/sample/sample-[md5].jpg
full: /data/[md5].[ext]
But how do I know if a sample is available?

Yeah, bring back the post count, please. Now I have to parse actual web pages to get even some kind of estimate how much posts did a search give, which is slower and generates more overhead traffic.

i hope i'm wrong, but based on https://github.com/r888888888/danbooru/issues/207 i would assume post count is too resource intensive and probably gone for good in v2.

but that's more than a year ago. there's a ticket in behalf of everyone.

Spiller said:
I have been unable to figure out how to look up a tag. /tags.xml?name=X doesn't work and /tags.xml?search=X errors (HTTP error 500).

Is there some problem with this?
http://danbooru.donmai.us/tags.xml?search[name_matches]=1girl

Spiller said:
I'm also a bit confused with the data. One tag has a name of "parent:712273", a post-count of 0 yet still have related tags. Another tag has a name of "tag:gypsy". (I could not find any such tags in my old copy of Danboorus tags either.)
/related_tag.json also gives me a error 500.

parent:712273 isn't an actual tag, but it does have posts, and the related tags are accurately showing what tags those posts have. I'm not sure if that's a bug.

tag:gypsy is a mistag with 0 posts. Empty tags can be hidden like this: http://danbooru.donmai.us/tags.xml?search[hide_empty]=yes
There's already an open issue about them not being excluded by default, so Albert will probably fix that soon.

Spiller said:
(Also fav-count seems always to be 0, no matter how long fav-string is?)

So that's why order:favcount is broken.

Spiller said:
But how do I know if a sample is available?

I'm not really a programmer, but couldn't you check if the <image-width> attribute is greater than 850? I believe Danbooru generates samples for anything at 851px width or greater.

Edit: To be more specific, fav-count hasn't updated since the upgrade to Danbooru 2. Favorites that occurred before it will be counted.

ghostrigger said:

i hope i'm wrong, but based on https://github.com/r888888888/danbooru/issues/207 i would assume post count is too resource intensive and probably gone for good in v2.

It was said that keeping track of deleted posts in relation to post count was "too much work". So we can't be sure post count is the same by its self. Secondly, "too much work" might mean that it would take more effort to implement than they are willing to do, not necessarily that it would be too expensive to have it running. (I have noticed this inaccuracy as well, I explicitly overwrite the post-count if I run out of results.)

If it is too expensive to calculate, why are they still willing to do it for their front-end? Presumingly, that is where the majority of their traffic is. (It only shows up if the amount of results are less than 20k though.)

But for now, you can still get it for "simple searches", where only one tag is provided. Just look up the tag instead and use its post-count. It is an extra request, but in my opinion it is worth it.

Toks said:

Is there some problem with this?
http://danbooru.donmai.us/tags.xml?search[name_matches]=1girl

Nope, works fine, it happens when I omit "[name_matches]". I never knew you had to do it that way, Rails continue to puzzle me...

Toks said:

I'm not really a programmer, but couldn't you check if the <image-width> attribute is greater than 850? I believe Danbooru generates samples for anything at 851px width or greater.

I could do that, but is that how it is defined in Danbooru? If I'm not completely sure about it, I will just end up with a buggy system. I would like to know, instead of just speculating. (Danbooru also resized if the hight is over 2000 px, but I could not find that in the v2.0 source.)
What worries me is that the API now becomes dependent on configurable settings, if someone sets up a Danbooru v2.0 board and fiddles with the settings, scripts will break.
It is not a bad thing that you have to deduce the wanted data using other data in an API, however there needs to be a clear definition on how to do this and the required data to do so must be easily obtainable, preferably programmatically.

Anyone know what the syntax is atm for the pools api? Been trying to get a list of posts based on pool ID.
http://danbooru.donmai.us/pools.xml is the link, json works as well, but can't get it to respond to anything else then "&pages=x"

itsonlyaname said:

Anyone know what the syntax is atm for the pools api? Been trying to get a list of posts based on pool ID.
http://danbooru.donmai.us/pools.xml is the link, json works as well, but can't get it to respond to anything else then "&pages=x"

http://danbooru.donmai.us/pools/(pool_id).xml

Posts are listed under <post-ids>.

Thanks :D just what i was looking for.
On another note, i still use http://danbooru.donmai.us/post/index.xml - will that one be removed in favor of http://danbooru.donmai.us/posts.xml ?

You'd need to ask Albert to know for sure, but I believe he stated the old API will only be deprecated, not removed entirely.

hmm,http://danbooru.donmai.us/pools/(pool_id).xml gets me all the ID's, which i can use in combination with http://danbooru.donmai.us/posts/(post_id).xml to get more details like the md5 (path), but that would mean sending a query for every image.

Where/how did you find the url for the pools if i may ask? might be able to solve my own problems instead of having to ask here every time :P

itsonlyaname said:
Where/how did you find the url for the pools if i may ask? might be able to solve my own problems instead of having to ask here every time :P

You mean http://danbooru.donmai.us/pools/(pool_id).xml?
I just guessed.

But it seems that going to the normal version of the page you want and adding .xml to the end always works.

So if you wanted this page:
http://danbooru.donmai.us/artists?order=date

You would just use:
http://danbooru.donmai.us/artists.xml?order=date

Same for .json.

I think this and the other information you're looking for is supposed to be written in help:api, but that page is broken, just like help:dtext and the others.

https://github.com/r888888888/danbooru/issues/643

Albert added a way of checking if a post has a sample, Spiller.

Any idea about the notes api? Specifically, a function to return all notes from a post.

(adding '.xml' before the '?' in these examples):
/note_versions?search[post_id]=1355274 - can search post_id specific, but is for the entire history of notes.
/notes?group_by=note&search[post_tags_match]=test - results are 'real' notes instead of history, but doesn't support a [post_id]= search
nothing with "&post_id=" seems to work either.

For API/scripting purposes, it'd be almost required to return all notes in a single call as well.

Edit; found some nice info at https://github.com/r888888888/danbooru/tree/master/app/models
but not enough to actually figure out something :/

Edit2; https://github.com/r888888888/danbooru/issues/754 - search[post_id] was added :)

Lightforger commented::

I only played with the notes API so far, but some of this info may be useful for the other API's.
Most of it is from poking around in the notes javascript: https://github.com/r888888888/danbooru/blob/master/app/assets/javascripts/notes.js

Listing notes: simply take the url from searching, and add .xml / .json ~
"/notes.xml?group_by=note&search[body_matches]=&search[post_tags_match]=&search[creator_name]="

Creating a note ~ works with or without .xml / .json
"/notes?note[x]=10&note[y]=10&note[width]=200&note[height]=50&note[body]=A+note&note[post_id]="
XMLHttpRequest method must be set to "POST"

Updating a note ~
same as creating, with extra argument: "&note[html_id]=" (ID of the note to update)
XMLHttpRequest method must be set to "PUT"

Deleting a note ~
"/notes/123456789.js"
XMLHttpRequest method must be set to "DELETE"

Fun fact: setting x/y to -1000 (which goes off the image) works, setting width to 7000(px) also works
Imagine notes of several million px high/wide :P

This worked for me:
with PUT:
"/posts/POST_ID.json" and possible arguments: &post[source]=&post[tag_string]=&post[rating]=
also noticed "post[old_tag_string]" and "post[is_note_locked]"

I'm trying to convert the Java program I used with the old API to the new. Retrieving post info is easy, but I'm not having any luck modifying tags/source of posts from the above. From the things I've tried, I've gotten an assortment of error codes: 403, 404, 406, 411, and 500. Anyone happen to have any ideas?

RaisingK said:

I've gotten an assortment of error codes: 403, 404, 406, 411, and 500. Anyone happen to have any ideas?

an innocent question, how do you login and ensure that you're authenticated by the server?

i noticed that the value of password_hash in the cookie is now different. the move towards bcrypt might have caused this.

and what credentials do we need to pass? do we still use login and password_hash variables? browsing the code i spotted user_name and password_hash. will these have any effect?

and maybe we can move the new api wiki by using the link already provided in the site_map so that others can see it as well. help:api?

I'm slowly updating help:api. I still have a couple resources left but you get the basic gist of the format.

good, api documentation is being built

There's a little something that bothers me, on pools api, the pool name has underscore, is it supposed to be like that?

*Daisu~ said:

good, api documentation is being built

There's a little something that bothers me, on pools api, the pool name has underscore, is it supposed to be like that?

Spaces are probably stored as underscores internally so that pool:pool_name syntax works for adding posts to pools.

oh okay

also, is the salt for the passwords still the same? (I guess it is)

*Daisu~ said:

oh okay

also, is the salt for the passwords still the same? (I guess it is)

Yep, still is the same

However, it think it's deprecated in favor of the new API key (found at 'my account' page).
See the Api wiki > Authentication for more details