Grafana influxdb error bad gateway

[Bug] Constant 502 Bad Gateway from Grafana After 4.0 Upgrade #6790

What Grafana version are you using?

What datasource are you using?

What OS are you running grafana on?

Ubuntu 16.04, behind nginx reverse proxy on same system.

Load a dashboard and leave it open for a few minutes.

Not have every panel error on refresh.

• What happened instead?
  After I leave any dashboard open for more than a few minutes, all panels get errors. Once I refresh the page, the panels load. Did not have any such problems before 4.0 upgrade.

An image or text representation of your metric query

aliasByNode(summarize(removeBelowValue(highestCurrent(sys.*.statsd.gauge-mygauge.total_something, 10), 1), ‘$Interval’, ‘avg’), 1)

The raw query and response for the network request

[pid 6684] write(139, «HTTP/1.0 502 Bad GatewayrnDate: Fri, 02 Dec 2016 00:28:56 GMTrnContent-Length: 0rnContent-Type: text/plain; charset=utf-8rnrn», 125

Grafana itself is returning 502 (not nginx), and even with logging set to Debug for file , there is no meaningful data or errors reported in /var/log/grafana/grafana.log . I realize there’s a lack of good debugging information here, but without logs, I’m not really sure how to narrow it down.

I’m okay with downgrading, but can’t find any documentation with regard to schema changes and if a downgrade would cause things to break. I can provide more debugging information if pointed in the right direction. Thanks!

The text was updated successfully, but these errors were encountered:

Источник

Network Error: Bad Gateway(502) #72

Iam getting this error I save the datasource:

Influxdb 1.7.10 (with flux enabled with INFLUXDB_HTTP_FLUX_ENABLED=true)
Grafana 6.6.2

The text was updated successfully, but these errors were encountered:

Ah seems influxdb is now using over 10GB of memory since I enabled flux. Bit much. This is causing it to crash.

Ah seems influxdb is now using over 10GB of memory since I enabled flux. Bit much. This is causing it to crash.

Hi Barsonax, did that solved your problem? I found myself in the same situations, how did you checked that influxdb is crashing when trying to connect with grafana?

Hi Barsonax, did that solved your problem?

I don’t see disabling flux as a proper solution.

I found myself in the same situations, how did you checked that influxdb is crashing when trying to connect with grafana?

I could see the it crashing in kubernetes because it was using too much memory. Memory usage was much lower without flux.

I could see the it crashing in kubernetes because it was using too much memory. Memory usage was much lower without flux.

Well, then at least I dont feel alone 🙂 It looks like it does many petitions and crashes as well through the web browser when doing a query through Grafana. But I am not sure if this is due influxdb v2, flux, grafana or the extension itself. Works but slow.

Did you took any other approach rather than disabling it? I was thinking in just using influxdb v1 and forget about flux.

not sure what we can do here. lets see if this is an issue with the 7x plugin

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

Bad gate way in Grafana when connecting to InfluxDB #289

Please run `microk8s.inspect
inspection-report-20190122_192827.tar.gz
We appreciate your feedback. Thank you for using microk8s.

I cannot get the influxdb to grafana. It always return 502 bad gateway error

The text was updated successfully, but these errors were encountered:

Can you please describe what you are deploying and how it fails? Can you share the manifests you are applying and/or instructions on how your setup is supposed to look like and how it is failing? I basically asking for a way to reproduce the issue you are seeing.

Hello, everybody. I have the same error. I only eval microk8s.enable prometheus and add the port forward to svc/grafana 3000 .

After you enable prometheus and having the pods running, can you access grafana through its ClusterIP (with something like http://10.152.183.153:3000/login ) ?

Can you help me reproduce this? How do you do the port forwarding? Are you running MicroK8s in a VM? Can you share the tarball from microk8s.inspect ? Thanks

Yes, I can connect to grafana and success login, but when I try to see metrics I got a red triangle in the corner of the graph. In network tab I found requests with 502 error code on url http://localhost:3000/api/datasources/proxy/1/api/v1/query?query=1%20-%20avg(rate(node_cpu_seconds_total%7Bmode%3D%22idle%22%7D%5B1m%5D))&time=1574896347 .

Port forwarding command: k port-forward -n kubectl monitoring svc/grafana 3000 . Then I forwarded the port to my computer using ssh.

Источник

HTTP Error Bad Gateway when using prometheus #14629

Read before posting:
Please include this information:

What Grafana version are you using?

What datasource are you using?

prometheus latest docke image

What OS are you running grafana on?

Linux 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

What did you do?

I create a prometheus datasource in grafana web ui

What was the expected result?

success create datasource

What happened instead?

Error with ‘HTTP Error Bad Gateway’

I run grafana and prometheus in docker with almost default configuration. after both started, i check connection in grafana:

i think it means that prometheus endpoint is ready for grafana backend.

then I creete data source in grafana ui like this:

so I think is a bug, or I miss some configuration?

The text was updated successfully, but these errors were encountered:

sorry for my mistake

@aximo Hi, did you find any solution for this problem?

I also suffer the same issue, let me know if you found a solution, thanks!

it’s networking / connection issue. Since it’s not a problem in Grafana we close it.

I have the same problem and if I curl the url from the Grafana back-end (pod) it works perfectly so it isn’t networking related

I’m new to Grafana & Prometheus my url is the /metrics uri on the service which works fine e.g.

I have done the ingress controller deployment from the helmchart which creates a custom metrics service that exposes 9913 on the ingress controller pod

I had the same problem and found that changing access from Server to Browser fixes it. It appears that the grafana server if being prevented from accessing the address:port while the browser is not blocked.

@torkelo I guess you could argue this is a network issue but you could also consider it a Grafana config issue. I prefer to use the Server option for better security.

I too was facing it. I’ve solved it but the solution is a bit hilarious. The solution is-
In the URL field, you need to explicitly write the URL (in my case it was http://localhost:9090). Earlier I was not writing it and there already was the url as watermark!

@souvikhaldar you are absolutely right. Its a water mark and not actual content. Thanks.

I still face this error after explicitly writing the URL (in my case it was http://localhost:9090), finally I resolved it by using http://prometheus:9090 or http:/172.17.0.2:9090 because localhost is not regonized as server host when I’m running both Prometheus and Grafana as containers and you should get the correct IP of prometheus container for grafana to connect to, below is the command line I used:

@jimmycgz Thank you for the answer. It now works.

Maybe it’s too late to answer but I faced the same problem and I solved it by using: host.docker.internal instead of localhost .

just a note, you typed host.internal.docker but it is actually host.docker.internal

I had the same issue being completely new to grafana, just following the tutorial. Changing from «Server» to «Browser» did the trick.

Had same issue. First time on grafana. Solved issue by using URL with IP address of Prometheus Docker Pod (docker inspect ) like so url: http://172.17.0.2:9090
This worked instantly.

I mapped port 8010 to 9090 when running Prometheus (-p 8010:9090).
It finds itself on the default network just as the Grafana container does.

I’m able to connect to it either with:
http://172.17.0.4:9090 (connects to the container directly)
or
http://172.17.0.1:8010 (connects to the mapped port)
or
http://server-ip:8010 (connects to the mapped port)

This solution worked for me

First check if port 9090 is working

sudo lsof -i:9090

then check the promethues logs

sudo docker-compose logs promethues

it showed me error global_scrape timeout greater than scrape_interval

they told to increase the scrape_timeout , but nothing about the scrape_interval in /root/bbb-monitoring/prometheus.yaml
So, I increased them to the same value

sudo docker-compose up -d

this should return

then I set the Data Source -> Promethues url to
http://localhost:9090

Hope this helps someone.

On running this on local minikube cluster, use the cluster address provided in logs to connect to prometheus server from grafana as the Access is Server(default) and Host clearly states : Your access method is Server, this means the URL needs to be accessible from the grafana backend/server.

I had the same problem and found that changing access from Server to Browser fixes it. It appears that the grafana server if being prevented from accessing the address:port while the browser is not blocked.

@torkelo I guess you could argue this is a network issue but you could also consider it a Grafana config issue. I prefer to use the Server option for better security.

@AndrewWPhillips changing the access to Browser worked for me. Thanks

Had same issue. First time on grafana. Solved issue by using URL with IP address of Prometheus Docker Pod (docker inspect ) like so url: http://172.17.0.2:9090
This worked instantly.

also works on your host IP like this HTTP://your_host_ip:9090, note that it’s your host IP, not «localhost», not «127.0.0.1»

use the cluster ip’s so http://172.20.207.29:9090

this is if grafana and prometeus are on kubernetes pods

exact solution for this and all related connection problems between docker images. Connection to prometheus to local app and from grafana to prometheus.
All of these issues is able to solved by using host.internal.docker instead of localhost or local ip

i think it’s problem http Access : select Browser not Server

I had the same issue being completely new to grafana, just following the tutorial. Changing from «Server» to «Browser» did the trick.

I did that but nothing shows up in dashboarh.

i had the same issue . In my case i think it happened because i was running graphana and prometheus in the same terminal so one would automatoically shutdown when the other will start. So obviouly graphana for example couldn’t connect with prometheus because it was off. When i open two different terminals and launched each application seperately it worked

The tip from @souvikhaldar was spot on. 🤦 . That is just an absurd hurdle for new users of Grafana to have to stumble across!
This github issue is Google’s first hit for «grafana bad gateway prometheus».

Please, @torkelo, if you could just put something other there, like «» or something, or actually let the help text there (http://localhost:9090/) be actual text instead, I guess you would help a lot of first time users like myself. @souvikhaldar’s comment currently has 47 👍 ‘s, and that are only the ones that admit to this being the problem!

Wrt. the docker containers — if you run both Prometheus and Grafana with —network=»host» (and then dropping the -p arguments for publishing the ports), it works with http://localhost:9090/ .

i think it’s problem http Access : select Browser not Server

this is saving my life 😀

I was facing this problem because I was trying to access localhost:9090 while Grafana was running in a docker container in a bridge network. 🙂

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

Connecting to Grafana — InfluxDB Error: Bad Request #20761

I am trying to view Influxdb2 data in Grafana. I use a python script to upload data into influxdb. This is confirmed to be working:

Influxdb is running on the IP shown in the image above. When I try to connect Grafana I get the error IInfluxDB Error: Bad Request

Following the steps in the documentation the url I input into grafana is the same one as in the image above.

The rest of the config is shown in the image below. Both passwords are the admin token and the database is my bucket ID.

Has anyone managed to get a minimum example working between Grafana and Influxdb2 or is there an obvious step I’m missing?

• System info: Linux 5.4.0-65-generic x86_64
• InfluxDB version: 2.0.4

The text was updated successfully, but these errors were encountered:

@C-monC I’m not familiar with the Grafana integration, but based on similar issues I’ve seen I’d recommend trying the instructions I listed at influxdata/kapacitor#2476 (comment) to expose your bucket via our V1 compatibility APIs. You’d then use:

• Database: name of the database you set for your DBRP resource
• Username: name you set for your V1 user
• Password: password you set for your V1 user

If that works for you, could you confirm here? I’ll make sure the Grafana-InfluxQL docs get updated to match the steps that work.

I’ve tried those steps — not sure which parameters must reference new objects and which should be new ones. For instance.
influx v1 dbrp create —org kubOrg —db testdb —rp rp —bucket-id d025cfae0bd77b14 -t asgsgsdfg
—db and —rp would be initialized by that name? Could not find a place to create them in the gui. The bucket is a bucket I’ve already made.

influx v1 auth create —org kubOrg —username test —password testtest —write-bucket d025cfae0bd77b14

Creating the token does not create a token in the gui but it does return a column which says «name/token»

ID Description Name / Token User Name User ID Permissions

In grafana I then add the datasource. Password testtest

The rest of the configuration — password is again testtest:

What happens if you switch «HTTP Method» to «POST»? We’ve seen strange failures when using GET (#20713)

If you have control over the server deployment, you could also run influxd —log-level debug to (hopefully) get some tracing logs on the server-side. If switching the HTTP Method doesn’t work, could you paste any logs you can find here?

Any update on this? Having same issue.

I have the same issue. Is there any newer Information?

Use Custom HTTP Header an not «Basic auth» Header = «Authorization» Value =»Token » important is the Space between the Word Token and your Token.

Use Custom HTTP Header an not «Basic auth» Header = «Authorization» Value =»Token » important is the Space between the Word Token and your Token.

I can confirm that this also fixed it for me. The token is created or copied from influxdb web console, then added as a custom http header in grafana datasource config as per instructions on the link above.

Use Custom HTTP Header an not «Basic auth» Header = «Authorization» Value =»Token » important is the Space between the Word Token and your Token.
Found on http://wiki.webperfect.ch/index.php?title=InfluxDB_2.x:_Error:_Bad_Request_(Grafana_and_InfluxQL)&oldid=2578

I can confirm that this also fixed it for me. The token is created or copied from influxdb web console, then added as a custom http header in grafana datasource config as per instructions on the link above.

This didn’t work for me. After generating a token and pasting it in Custom HTTP Headers with the right format (space between «Token» and ) I’m still getting a Bad Request error.

Use Custom HTTP Header an not «Basic auth» Header = «Authorization» Value =»Token » important is the Space between the Word Token and your Token.

I was having the exact same issue and this fix also worked for me. I also continued using the GET http method, not sure if that had anything to do with it.

The process for connecting Grafana to InfluxDB 2.x and using InfluxQL is documented here: https://docs.influxdata.com/influxdb/v2.1/tools/grafana/?t=InfluxQL

Issue resolved for me as well

Format will be like : Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note: You have to write token space and then actual token value

Use Custom HTTP Header an not «Basic auth» Header = «Authorization» Value =»Token » important is the Space between the Word Token and your Token.

You can also roll back to v1.8.4 to fix this problem.
https://stackoverflow.com/a/66732230/8712494

Use Custom HTTP Header an not «Basic auth» Header = «Authorization» Value =»Token » important is the Space between the Word Token and your Token.

You absolute legend. Thank you! That space makes complete sence, just annoying how the value is star’d out so you think it’s just the API key it required! «header» «token api_key» with a space. You sir, are amazing.

Wow, I’m shocked this worked. Great job.

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

When I tried the tutorial that came out by googled «Grafana InfluxDB Docker», the story that was troubled by the above Bad Gateway.
The bottom line was that I didn’t understand the combination of URL and Access correctly.

Implementation environment

I launched a Grafana and InfluxDB container on Docker on my local PC and tried to configure InflxDB from Grafana’s Data Sources.
Write the following Docker-compose.yml and execute it.

version: "3"
services:
  influxdb:
    image: influxdb:latest
    ports:
      - "8086:8086"
    volumes:
      - ./data/influxdb:/var/lib/influxdb

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    volumes:
      - ./data/grafana:/var/lib/grafana
    depends_on:
      - influxdb

I like volumes for feasibility, but this time I will create a «data» folder in the same hierarchy as docker-compose.yml, and create an «influxdb» folder and a «grafana» folder under it.

Access InfluxDB with docker command, create a new database (sample in this case) and measurement, then access Grafana with " http: // localhost: 3000 " and Data Sources of Configuration When I entered the information to add more InfluxDB and even executed «Save & Test», the «InfluxDB Error: Bad Gateway» as shown at the beginning occurred.

Solution (1) Change Access from Server to Browser (not recommended)

This is the solution that I was able to connect while playing with various settings.
If you understand the meaning after that, this ① is deprecated and you should carry out ② described later, so it is only for reference.

If you want to establish a connection with the URL " https: // localhost: 8086 ", change Access from » Server « to » Browser « after» Save & Test «. , «Data source is working» and now connected.

By the way, what is this Access?
If you select «Help» on the right, the following description will be displayed.

In summary, I understand that:
-** Server ** attempts access via Grafana’s backend (internal communication)
-** Browser ** tries to access directly from the browser (external communication)

In other words, the reason I stumbled this time is that the URL is «localhost: 8086» when ** Server **, so ** I tried to access the 8086 port on the Grafana container **, so Bad Gateway occurred. It will be.

However, in solution (1), localhost (local PC itself) is now accessed as external communication instead of internal communication, so it is possible to connect.
However, in actual cases, there are almost no cases where the database layer like InfluxDB can be accessed from the outside, and it is desirable to be able to access the DB by internal communication.

Therefore, this time, I made it accessible with the following solution.

Solution (2) Change the URL to a name (address) for internal communication

This time, since the Grafana container and InfluxDB container are started from Docker compose, it depends on the description of docker-compose.yml and the directory name, but this time it will be in the following state as an example.

 $ docker-compose ps
           Name                      Command           State           Ports
-------------------------------------------------------------------------------------
influxdb_grafana_grafana_1    /run.sh                  Up      0.0.0.0:3000->3000/tcp
influxdb_grafana_influxdb_1   /entrypoint.sh influxd   Up      0.0.0.0:8086->8086/tcp

In the above case, the container name of InfluxDB will be «influxdb_grafana_influxdb_1».
In docker-compose, name resolution can be done by default with each other’s container name from network, so by using this «influxdb_grafana_influxdb_1» as the URL name, Access will be able to connect even with ** Server **.
In this example, the URL is http: // influxdb_grafana_influxdb_1: 8086.